Moving picture coding method and moving picture decoding method for performing inter picture prediction coding and inter picture prediction decoding using previously processed pictures as reference pictures

ABSTRACT

A coding control unit ( 110 ) and a mode selection unit ( 109 ) are included. The coding control unit ( 110 ) determines the coding order for a plurality of consecutive B-pictures located between I-pictures and P-pictures so that the B-picture whose temporal distance from two previously coded pictures is farthest in display order is coded by priority, so as to reorder the B-pictures in coding order. When a current block is coded in direct mode, the mode selection unit  109  scales a forward motion vector of a block which is included in a backward reference picture of a current picture and co-located with the current block, so as to generate motion vectors of the current block, if the forward motion vector has been used for coding the co-located block.

This application is a divisional application of application Ser. No.10/468,119, filed Aug. 15, 2003, now U.S. Pat. No. 7,664,180 which isthe National Stage of International Application No. PCT/JP03/02099,filed Feb. 26, 2003.

TECHNICAL FIELD

The present invention relates to moving picture coding methods andmoving picture decoding methods, and particularly to methods forperforming inter picture prediction coding and inter picture predictiondecoding of a current picture using previously processed pictures asreference pictures.

BACKGROUND ART

In moving picture coding, data amount is generally compressed byutilizing the spatial and temporal redundancies that exist within amoving picture. Generally speaking, frequency transformation is used asa method utilizing the spatial redundancies, and inter pictureprediction coding is used as a method utilizing the temporalredundancies. In the inter picture prediction coding, for coding acurrent picture, previously coded pictures earlier or later than thecurrent picture in display order are used as reference pictures. Theamount of motion of the current picture from the reference picture isestimated, and the difference between the picture data obtained bymotion compensation based on that amount of motion and the picture dataof the current picture is calculated, so that the temporal redundanciesare eliminated. The spatial redundancies are further eliminated fromthis differential value so as to compress the data amount of the currentpicture.

In the moving picture coding method called H.264 which has beendeveloped for standardization, a picture which is coded not using interpicture prediction but using intra picture coding is referred to as anI-picture, a picture which is coded using inter picture prediction withreference to one previously processed picture which is earlier or laterthan a current picture in display order is referred to as a P-picture,and a picture which is coded using inter picture prediction withreference to two previously processed pictures which are earlier orlater than a current picture in display order is referred to as aB-picture (See ISO/IEC 14496-2 “Information technology—Coding ofaudio-visual objects—Part 2: Visual” pp. 218-219).

FIG. 1A is a diagram showing relationship between respective picturesand the corresponding reference pictures in the above-mentioned movingpicture coding method, and FIG. 1B is a diagram showing the sequence ofthe pictures in the bit stream generated by coding.

A picture I1 is an I-picture, pictures P5, P9 and P13 are P-pictures,and pictures B2, B3, B4, B6, B7, B8, B10, B11 and B12 are B-pictures. Asshown by the arrows, the P-pictures P5, P9 and P13 are coded using interpicture prediction from the I-picture I1 and P-pictures P5 and P9respectively as reference pictures.

As shown by the arrows, the B-pictures B2, B3 and B4 are coded usinginter picture prediction from the I-picture I1 and P-picture P5respectively as reference pictures. In the same manner, the B-picturesB6, B7 and B8 are coded using the P-pictures P5 and P9 respectively asreference pictures, and the B-pictures B10, B11 and B12 are coded usingthe P-pictures P9 and P13 respectively as reference pictures.

In the above-mentioned coding, the reference pictures are coded prior tothe pictures which refer to the reference pictures. Therefore, the bitstream is generated by the above coding in the sequence as shown in FIG.1B.

By the way, in the H.264 moving picture coding method, a coding modecalled direct mode can be selected. An inter picture prediction methodin direct mode will be explained with reference to FIG. 2. FIG. 2 is anillustration showing motion vectors in direct mode, and particularlyshowing the case of coding a block a in the picture B6 in direct mode.In this case, a motion vector c used for coding a block b in the pictureP9 is utilized. The block b is co-located with the block a and thepicture P9 is a backward reference picture of the picture B6. The motionvector c is a vector used for coding the block b and refers to thepicture P5. The block a is coded using bi-prediction based on thereference blocks obtained from the forward reference picture P5 and thebackward reference picture P9 using vectors parallel to the motionvector c. In other words, the motion vectors used for coding the block aare the motion vector d for the picture P5 and the motion vector e forthe picture P9.

However, when B-pictures are coded using inter picture prediction withreference to I and P-pictures, the temporal distance between the currentB-picture and the reference picture may be long, which causes reductionof coding efficiency. Particularly when a lot of B-pictures are locatedbetween adjacent I-picture and P-picture or two P-pictures closest toeach other, coding efficiency is significantly reduced.

The present invention has been conceived in order to solve theabove-mentioned problem, and it is an object of the present invention toprovide a moving picture coding method and a moving picture decodingmethod for avoiding efficiency reduction of coding B-pictures if a lotof B-pictures are located between an I-picture and a P-picture orbetween two P-pictures. In addition, it is another object to provide amoving picture coding method and a moving picture decoding method forimproving coding efficiency in direct mode.

DISCLOSURE OF INVENTION

In order to achieve above-mentioned object, the moving picture codingmethod of the present invention is a moving picture coding method forcoding picture data corresponding to pictures that form a moving pictureand generating a bit stream, the moving picture coding methodcomprising: a coding step for coding a current picture as one of anI-picture, a P-picture and a B-picture, the I-picture having only blockswhich are intra picture coded, the P-picture having a block which isinter picture prediction coded with uni-predictive reference using apreviously coded picture as a first reference picture, and the B-picturehaving a block which is inter picture prediction coded withbi-predictive reference using previously coded pictures as a firstreference picture and a second reference picture, wherein the codingstep includes a control step for determining coding order which isdifferent from display order for consecutive B-pictures located betweenI-pictures and P-pictures.

Therefore, since B-pictures can be coded using pictures which aretemporally closer in display order as reference pictures, predictionefficiency for motion compensation is improved and thus codingefficiency can be increased.

Also, the moving picture coding method according to the presentinvention is a moving picture coding method for coding picture datacorresponding to pictures that form a moving picture and generating abit stream, the moving picture coding method comprising: a coding stepfor coding a current picture as a B-picture having a block which isinter picture prediction coded with bi-predictive reference usingpreviously coded pictures as a first reference picture and a secondreference picture, wherein in the coding step, when a current block A ina current B-picture is coded in direct mode by which motion compensationof the current block A is performed using motion vectors of the currentblock A obtained from a motion vector of a previously coded block, themotion vectors for performing the motion compensation of the currentblock A are obtained by scaling a first motion vector, based on a firstreference picture, of a co-located block B in the second referencepicture of the current block A, using a difference specified byinformation indicating display order of pictures.

Therefore, when the direct mode is selected, since a first motion vectorof a second reference picture is scaled, there is no need to add motionvector information to a bit stream, and prediction efficiency can alsobe improved.

Likewise, when a current block A in a current B-picture is coded indirect mode, the motion vectors for performing the motion compensationof the current block A may be obtained by scaling a second motionvector, based on a second reference picture, of a co-located block B inthe second reference picture of the current block A, using a differencespecified by information indicating display order of pictures.

Therefore, when the direct mode is selected, since a second motionvector of a second reference picture is scaled, there is no need to addmotion vector information to a bit stream, and prediction efficiency canalso be improved.

Furthermore, when a current block A in a current B-picture is coded indirect mode, if a co-located block B in the second reference picture ofthe current block A is previously coded in direct mode, the motionvectors for performing the motion compensation of the current block Amay be obtained by scaling a first motion vector, based on a firstreference picture of the block B, substantially used for coding theblock B in the second reference picture, using a difference specified byinformation indicating display order of pictures.

Therefore, when the direct mode is selected, since a first motion vectorof a second reference picture which has been substantially used forcoding the second reference picture is scaled, there is no need to addmotion vector information to a bit stream, and prediction efficiency canalso be improved.

Also, when a current block A in a current B-picture is coded in directmode, the motion vectors for performing the motion compensation of thecurrent block A may be obtained by scaling a first motion vector, basedon a first reference picture, of a co-located block B in a temporallylater P-picture, using a difference specified by information indicatingdisplay order of pictures.

Therefore, when the direct mode is selected, since a first motion vectorof a temporally later P-picture is scaled, there is no need to addmotion vector information to a bit stream, and prediction efficiency canalso be improved.

Furthermore, when a current block A in a current B-picture is coded indirect mode, the motion vectors for performing the motion compensationof the current block A may be obtained by scaling a first motion vectorif a co-located block B in the second reference picture of the currentblock A is coded using at least the first motion vector based on a firstreference picture of the block B, and scaling a second motion vector ifthe block B is coded using only the second motion vector based on asecond reference picture of the block B, using a difference specified byinformation indicating display order of pictures.

Therefore, when the direct mode is selected, if a second referencepicture has a first motion vector, this first motion vector is scaled,and if the second reference picture does not have a first motion vectorbut only a second motion vector, this second motion vector is scaled.So, there is no need to add motion vector information to a bit stream,and prediction efficiency can be improved.

In addition, the moving picture decoding method according to the presentinvention is a moving picture decoding method for decoding a bit streamwhich is generated by coding picture data corresponding to pictures thatform a moving picture, the moving picture decoding method comprising: adecoding step for decoding a current picture by inter picture predictionusing a previously decoded picture as a reference picture, wherein inthe decoding step, when the current picture is decoded by the interpicture prediction with bi-predictive reference using the previouslydecoded pictures as a first reference picture and a second referencepicture, a bit stream including at least a picture which is temporallyclosest to the current picture in display order, as the first referencepicture or the second reference picture, is decoded.

Therefore, a bit stream, which is generated by coding a picture by interpicture prediction with bi-predictive reference using pictures which aretemporally close in display order as a first reference picture and asecond reference picture, can be properly decoded.

Also, the moving picture decoding method according to the presentinvention is a moving picture decoding method for decoding a bit streamwhich is generated by coding picture data corresponding to pictures thatform a moving picture, the moving picture decoding method comprising: adecoding step for decoding a current picture by inter picture predictionusing a previously decoded picture as a reference picture, wherein inthe decoding step, when the current picture is a picture having a blockwhich is decoded by inter picture prediction with bi-predictivereference using previously decoded pictures as a first reference pictureand a second reference picture, and a current block A is decoded indirect mode by which motion compensation of the current block A isperformed using motion vectors of the current block A obtained from amotion vector of a previously decoded block, the motion vectors forperforming the motion compensation of the current block A are obtainedby scaling a first motion vector, based on a first reference picture, ofa co-located block B in the second reference picture of the currentblock A, using a difference specified by information indicating displayorder of pictures.

Therefore, when the direct mode is selected, since a first motion vectorof a second reference picture is scaled, proper decoding can beachieved.

Likewise, when a current picture is a picture having a block which isdecoded by inter picture prediction with bi-predictive reference and acurrent block A is decoded in direct mode, the motion vectors forperforming the motion compensation of the current block A may beobtained by scaling a second motion vector, based on a second referencepicture, of a co-located block B in the second reference picture of thecurrent block A, using a difference specified by information indicatingdisplay order of pictures.

Therefore, when the direct mode is selected, since a second motionvector of a second reference picture is scaled, proper decoding can beachieved.

Furthermore, when a current picture is a picture having a block which isdecoded by inter picture prediction with bi-predictive reference and acurrent block A is decoded in direct mode, if a co-located block B inthe second reference picture of the current block A is previouslydecoded in direct mode, the motion vectors for performing the motioncompensation of the current block A may be obtained by scaling a firstmotion vector, based on a first reference picture of the block B,substantially used for decoding the block B in the second referencepicture, using a difference specified by information indicating displayorder of pictures.

Therefore, when the direct mode is selected, since a first motion vectorof a second reference picture which has been substantially used fordecoding the second reference picture is scaled, proper decoding can beachieved.

Also, when a current picture is a picture having a block which isdecoded by inter picture prediction with bi-predictive reference and acurrent block A is decoded in direct mode, the motion vectors forperforming the motion compensation of the current block A may beobtained by scaling a first motion vector, based on a first referencepicture, of a co-located block B in a temporally later picture, using adifference specified by information indicating display order ofpictures, the later picture being inter picture prediction decoded withuni-predictive reference using a previously decoded picture as a firstreference picture.

Therefore, when the direct mode is selected, since a first motion vectorof a picture which is decoded by inter picture prediction withuni-predictive reference is scaled, proper decoding can be achieved.

The present invention can be realized as such a moving picture codingmethod and a moving picture decoding method as mentioned above, but alsoas a moving picture coding apparatus and a moving picture decodingapparatus including characteristic steps of these moving picture codingmethod and moving picture decoding method. In addition, the presentinvention can be realized as a bit stream obtained by coding by themoving picture coding method so as to distribute it via a recordingmedium such as a CD-ROM or a transmission medium such as the Internet.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic diagram showing prediction relations betweenpictures and their sequence in the conventional moving picture codingmethod, and 1A shows the relations between respective pictures and thecorresponding reference pictures, and FIG. 1B shows the sequence of thepictures in a bit stream generated by coding.

FIG. 2 is a schematic diagram showing motion vectors in direct mode inthe conventional moving picture coding method.

FIG. 3 is a block diagram showing the structure of a first embodiment ofa moving picture coding apparatus using a moving picture coding methodaccording to the present invention.

FIG. 4 is an illustration of picture numbers and relative indices in theembodiments of the present invention.

FIG. 5 is a conceptual illustration of a moving picture coded dataformat in the moving picture coding apparatus in the embodiments of thepresent invention.

FIG. 6 is an illustration showing the picture sequence in a reorderingmemory in the embodiments of the present invention, and FIG. 6A showsthe sequence in input order, and FIG. 6B shows the reordered sequence.

FIG. 7 is a schematic diagram showing motion vectors in direct mode inthe embodiments of the present invention, and FIG. 7A shows a case wherea current block a is a picture B7, FIG. 7B shows first and secondexamples in a case where a current block a is a picture B6, FIG. 7Cshows a third example in a case where a current block a is a picture B6,and FIG. 7D shows a fourth example in a case where a current block a isa picture B6.

FIG. 8 is a schematic diagram showing motion vectors in direct mode inthe embodiments of the present invention, and FIG. 8A shows a fifthexample in a case where a current block a is a picture B6, FIG. 8B showsa sixth example in a case where a current block a is a picture B6, FIG.8C shows a seventh example in a case where a current block a is apicture B6, and FIG. 8D shows a case where a current block a is apicture B8.

FIG. 9 is a schematic diagram showing prediction relations betweenrespective pictures and their sequence in the embodiments of the presentinvention, and FIG. 9A shows the prediction relations between respectivepictures indicated in display order, and FIG. 9B shows the sequence ofthe pictures reordered in coding order (in a bit stream).

FIG. 10 is a schematic diagram showing prediction relations betweenrespective pictures and their sequence in the embodiments of the presentinvention, and FIG. 10A shows the prediction relations betweenrespective pictures indicated in display order, and FIG. 10B shows thesequence of the pictures reordered in coding order (in a bit stream).

FIG. 11 is a schematic diagram showing prediction relations betweenrespective pictures and their sequence in the embodiments of the presentinvention, and FIG. 10A shows the prediction relations betweenrespective pictures indicated in display order, and FIG. 10B shows thesequence of the pictures reordered in coding order (in a bit stream).

FIG. 12 is a schematic diagram showing hierarchically the pictureprediction structure as shown in FIG. 6 in the embodiments of thepresent invention.

FIG. 13 is a schematic diagram showing hierarchically the pictureprediction structure as shown in FIG. 9 in the embodiments of thepresent invention.

FIG. 14 is a schematic diagram showing hierarchically the pictureprediction structure as shown in FIG. 10 in the embodiments of thepresent invention.

FIG. 15 is a schematic diagram showing hierarchically the pictureprediction structure as shown in FIG. 11 in the embodiments of thepresent invention.

FIG. 16 is a block diagram showing the structure of an embodiment of amoving picture decoding apparatus using a moving picture decoding methodaccording to the present invention.

FIG. 17 is an illustration of a recording medium for storing a programfor realizing the moving picture coding method and the moving picturedecoding method in the first and second embodiments by a computersystem, and FIG. 17A shows an example of a physical format of a flexibledisk as a body of recording medium, FIG. 17B shows a cross-sectionalview and a front view of the appearance of the flexible disk and theflexible disk itself, FIG. 17C shows a structure for recording andreproducing the program on the flexible disk FD.

FIG. 18 a block diagram showing the overall configuration of a contentsupply system for realizing content distribution service.

FIG. 19 is a sketch showing an example of a mobile phone.

FIG. 20 is a block diagram showing the internal structure of the mobilephone.

FIG. 21 is a block diagram showing the overall configuration of adigital broadcast system.

BEST MODE FOR CARRYING OUT THE INVENTION

The embodiments of the present invention will be explained below withreference to the figures.

First Embodiment

FIG. 3 is a block diagram showing the structure of an embodiment of themoving picture coding apparatus using the moving picture coding methodaccording to the present invention.

As shown in FIG. 3, the moving picture coding apparatus includes areordering memory 101, a difference calculation unit 102, a residualerror coding unit 103, a bit stream generation unit 104, a residualerror decoding unit 105, an addition unit 106, a reference picturememory 107, a motion vector estimation unit 108, a mode selection unit109, a coding control unit 110, switches 111˜115 and a motion vectorstorage unit 116.

The reordering memory 101 stores moving pictures inputted on apicture-to-picture basis in display order. The coding control unit 110reorders the pictures stored in the reordering memory 101 in codingorder. The coding control unit 110 also controls the operation of themotion vector storage unit 116 for storing motion vectors.

Using the previously coded and decoded picture data as a referencepicture, the motion vector estimation unit 108 estimates a motion vectorindicating a position which is predicted optimum in the search area inthe reference picture. The mode selection unit 109 determines a mode forcoding macroblocks using the motion vector estimated by the motionvector estimation unit 108, and generates predictive image data based onthe coding mode. The difference calculation unit 102 calculates thedifference between the image data read out from the reordering memory101 and the predictive image data inputted by the mode selection unit109, and generates residual error image data.

The residual error coding unit 103 performs coding processing such asfrequency transform and quantization on the inputted residual errorimage data for generating the coded data. The bit stream generation unit104 performs variable length coding or the like on the inputted codeddata, and further adds the motion vector information, the coding modeinformation and other relevant information inputted by the modeselection unit 109 to the coded data so as to generate a bit stream.

The residual error decoding unit 105 performs decoding processing suchas inverse quantization and inverse frequency transform on the inputtedcoded data for generating decoded differential image data. The additionunit 106 adds the decoded differential image data inputted by theresidual error decoding unit 105 and the predictive image data inputtedby the mode selection unit 109 for generating decoded image data. Thereference picture memory 107 stores the generated decoded image data.

FIG. 4 is an illustration of pictures and relative indices. The relativeindices are used for identifying uniquely reference pictures stored inthe reference picture memory 107, and they are associated to respectivepictures as shown in FIG. 4. The relative indices are also used forindicating the reference pictures which are to be used for coding blocksusing inter picture prediction.

FIG. 5 is a conceptual illustration of moving picture coded data formatused by the moving picture coding apparatus. Coded data “Picture” forone picture includes header coded data “Header” included in the head ofthe picture, block coded data “Block1” for direct mode, block coded data“Block2” for the inter picture prediction other than the direct mode,and the like. The block coded data “Block2” for the inter pictureprediction other than direct mode has a first relative index “RIdx1” anda second relative index “RIdx2” for indicating two reference picturesused for inter picture prediction, a first motion vector “MV1” and asecond motion vector “MV2” in this order. On the other hand, the blockcoded data “Block1” for direct mode does not have the first and secondrelative indices “RIdx1” and “RIdx2” and the first and second motionvectors “MV1” and “MV2”. The index which is to be used, the firstrelative index “RIdx1” or the second relative index “RIdx2”, can bedetermined by the prediction type “PredType”. Also, the first relativeindex “RIdx1” indicates a first reference picture, and the secondrelative index “RIdx2” indicates a second reference picture. In otherwords, whether a picture is a first reference picture or a secondreference picture is determined based on where they are located in thebit stream.

Note that a P-picture is coded by inter picture prediction withuni-predictive reference using a previously coded picture which islocated earlier or later in display order as a first reference picture,and a B-picture is coded by inter picture prediction with bi-predictivereference using previously coded pictures which are located earlier orlater in display order as a first reference picture and a secondreference picture. In the first embodiment, the first reference pictureis explained as a forward reference picture, and the second referencepicture is explained as a backward reference picture. Furthermore, thefirst and second motion vectors for the first and second referencepictures are explained as a forward motion vector and a backward motionvector respectively.

Next, how to assign the first and second relative indices will beexplained with reference to FIG. 4A.

As the first relative indices, in the information indicating displayorder, the values incremented by 1 from 0 are first assigned to thereference pictures earlier than the current picture from the picturecloser to the current picture. After the values incremented by 1 from 0are assigned to all the reference pictures earlier than the currentpicture, then the subsequent values are assigned to the referencepictures later than the current picture from the picture closer to thecurrent picture.

As the second relative indices, in the information indicating displayorder, the values incremented by 1 from 0 are assigned to the referencepictures later than the current picture from the picture closer to thecurrent picture. After the values incremented by 1 from 0 are assignedto all the reference pictures later than the current picture, then thesubsequent values are assigned to the reference pictures earlier thanthe current picture from the picture closer to the current picture.

For example, in FIG. 4A, when the first relative index “RIdx1” is 0 andthe second relative index “RIdx2” is 1, the forward reference picture isthe B-picture No. 6 and the backward reference picture is the P-pictureNo. 9. Here, these picture numbers 6 and 9 indicate the display order.

Relative indices in a block are represented by variable length codewords, and the codes with shorter lengths are assigned to the indices ofthe smaller values. Since the picture which is closest to the currentpicture is usually selected as a reference picture for inter pictureprediction, coding efficiency is improved by assigning the relativeindex values in order of closeness to the current picture.

Assignment of reference pictures to relative indices can be changedarbitrarily if it is explicitly indicated using buffer control signal incoded data (RPSL in Header as shown in FIG. 5). This enables to changethe reference picture with the second relative index “0” to an arbitraryreference picture in the reference picture memory 107. As shown in FIG.4B, assignment of reference indices to pictures can be changed, forexample.

Next, the operation of the moving picture coding apparatus structured asabove will be explained below.

FIG. 6 is an illustration showing the picture sequence in the reorderingmemory 101, and FIG. 6A shows the sequence in input order and FIG. 6Bshows the reordered sequence. Here, vertical lines show pictures, andthe numbers indicated at the lower right of the pictures show thepicture types (I, P and B) with the first alphabetical letters and thepicture numbers indicating display order with the following numbers.

As shown in FIG. 6A, a moving picture is inputted to the reorderingmemory 101 on a picture-to-picture basis in display order, for example.When the pictures are inputted to the reordering memory 101, the codingcontrol unit 110 reorders the pictures inputted to the reordering memory101 in coding order. The pictures are reordered based on the referencerelations in inter picture prediction coding, and more specifically, thepictures are reordered so that the pictures used as reference picturesare coded earlier than the pictures which use the reference pictures.

Here, it is assumed that a P-picture refers to one neighboringpreviously processed I or P-picture which is located earlier or laterthan the current P-picture in display order, and a B-picture refers totwo neighboring previously processed pictures which are located earlieror later than the current B-picture in display order.

The pictures are coded in the following order. First, a B-picture at thecenter of B-pictures (3 B-pictures in FIG. 6A, for instance) locatedbetween two P-pictures is coded, and then another B-picture closer tothe earlier P-picture is coded. For example, the pictures B6, B7, B8 andP9 are coded in the order of P9, B7, B6 and B8.

In this case, in FIG. 6A, the picture pointed by the arrow refers to thepicture at the origin of the arrow. Specifically, B-picture B7 refers toP-pictures P5 and P9, B6 refers to P5 and B7, and B8 refers to B7 andP9, respectively. The coding control unit 110 reorders the pictures incoding order, as shown in FIG. 6B.

Next, the pictures reordered in the reordering memory 101 are read outin a unit for every motion compensation. Here, the unit of motioncompensation is referred to as a macroblock which is 16 (horizontal)×16(vertical) pixels in size. Coding of the pictures P9, B7 B6 and B8 shownin FIG. 6A will be explained below in this order.

(Coding of Picture P9)

The P-picture P9 is coded using inter picture prediction with referenceto one previously processed picture located earlier or later than P9 indisplay order. In coding P9, the picture P5 is the reference picture, asmentioned above. P5 has already been coded and the decoded picturethereof is stored in the reference picture memory 107. In codingP-pictures, the coding control unit 110 controls switches 113, 114 and115 so as to be ON. The macroblocks in the picture P9 read out from thereordering memory 101 are thus inputted to the motion vector estimationunit 108, the mode selection unit 109 and the difference calculationunit 102 in this order.

The motion vector estimation unit 108 estimates a motion vector of amacroblock in the picture P9, using the decoded picture data of thepicture P5 stored in the reference picture memory 107 as a referencepicture, and outputs the estimated motion vector to the mode selectionunit 109.

The mode selection unit 109 determines the mode for coding themacroblock in the picture P9 using the motion vector estimated by themotion vector estimation unit 108. Here, the coding mode indicates themethod of coding macroblocks. As for P-pictures, it determines any ofthe coding methods, intra picture coding, inter picture predictioncoding using a motion vector and inter picture prediction coding withoutusing a motion vector (where motion is handled as “0”). For determininga coding mode, a method is selected so that a coding error is reducedwith a small amount of bits.

The mode selection unit 109 outputs the determined coding mode to thebit stream generation unit 104. If the coding mode determined by themode selection unit 109 is inter picture prediction coding, the motionvector which is to be used for the inter picture prediction coding isoutputted to the bit stream generation unit 104 and further stored inthe motion vector storage unit 116.

The mode selection unit 109 generates predictive image data based on thedetermined coding mode for generating to the difference calculation unit102 and the addition unit 106. However, when selecting intra picturecoding, the mode selection unit 109 does not output predictive imagedata. In addition, when selecting intra picture coding, the modeselection unit 109 controls the switches 111 and 112 to connect to “a”side and “c” side respectively, and when selecting inter pictureprediction coding, it controls them to connect to “b” side and “d” siderespectively. The case will be explained below where the mode selectionunit 109 selects inter picture prediction coding.

The difference calculation unit 102 receives the image data of themacroblock in the picture P9 read out from the reordering memory 101 andthe predictive image data outputted from the mode selection unit 109.The difference calculation unit 102 calculates the difference betweenthe image data of the macroblock in the picture P9 and the predictiveimage data, and generates the residual error image data for outputtingto the residual error coding unit 103.

The residual error coding unit 103 performs coding processing such asfrequency transform and quantization on the inputted residual errorimage data and thus generates the coded data for outputting to the bitstream generation unit 104 and the residual error decoding unit 105.Here, the coding processing such as frequency transform and quantizationis performed in every 8 (horizontal)×8 (vertical) pixels or 4(horizontal)×4 (vertical) pixels, for example.

The bit stream generation unit 104 performs variable length coding orthe like on the inputted coded data, and further adds information suchas motion vectors and a coding mode, header information and so on to thecoded data for generating and outputting the bit stream.

On the other hand, the residual error decoding unit 105 performsdecoding processing such as inverse quantization and inverse frequencytransform on the inputted coded data and generates the decodeddifferential image data for outputting to the addition unit 106. Theaddition unit 106 adds the decoded differential image data and thepredictive image data inputted by the mode selection unit 109 forgenerating the decoded image data, and stores it in the referencepicture memory 107.

That is the completion of coding one macroblock in the picture P9.According to the same processing, the remaining macroblocks of thepicture P9 are coded. And after all the macroblocks of the picture P9are coded, the picture B7 is coded.

(Coding of Picture B7)

The picture B7 refers to the picture P5 as a forward reference pictureand the picture P9 as a backward reference picture. Since the picture B7is used as a reference picture for coding other pictures, the codingcontrol unit 110 controls the switches 113, 114 and 115 so as to be ON,which causes the macroblocks in the picture B7 read out from thereordering memory 101 to be inputted to the motion vector estimationunit 108, the mode selection unit 109 and the difference calculationunit 102.

Using the decoded picture data of the picture P5 and the decoded picturedata of the picture P9 which are stored in the reference picture memory107 as a forward reference picture and a backward reference picturerespectively, the motion vector estimation unit 108 estimates a forwardmotion vector and a backward motion vector of the macroblock in thepicture B7. And the motion vector estimation unit 108 outputs theestimated motion vectors to the mode selection unit 109.

The mode selection unit 109 determines the coding mode for themacroblock in the picture B7 using the motion vectors estimated by themotion vector estimation unit 108. Here, it is assumed that a codingmode for B-pictures can be selected from among intra picture coding,inter picture prediction coding using a forward motion vector, interpicture prediction coding using a backward motion vector, inter pictureprediction coding using bi-predictive motion vectors and direct mode.

Operation of direct mode coding will be explained with reference to FIG.7A. FIG. 7A is an illustration showing motion vectors in direct mode,and specifically shows the case where the block a in the picture B7 iscoded in direct mode. In this case, a motion vector c, which has beenused for coding the block b in the picture P9, is utilized. The block bis co-located with the block a, and the picture P9 is a backwardreference picture of the picture B7. The motion vector c is stored inthe motion vector storage unit 116. The block a is bi-predicted from theforward reference picture P5 and the backward reference picture P9 usingvectors obtained utilizing the motion vector c. For example, as a methodof utilizing the motion vector c, there is a method of generating motionvectors parallel to the motion vector c. In this case, the motion vectord and the motion vector e are used for the picture P5 and the picture P9respectively for coding the block a.

In this case where the forward motion vector d is MVF, the backwardmotion vector e is MVB, the motion vector c is MV, the temporal distancebetween the backward reference picture P9 for the current picture B7 andthe picture P5 which the block in the backward reference picture P9refers to is TRD, and the temporal distance between the current pictureB7 and the forward reference picture P5 is TRF respectively, the motionvector d MVF and the motion vector e MVB are respectively calculated byEquation 1 and Equation 2. Note that the temporal distance between thepictures can be determined based on the information indicating thedisplay order (position) given to the respective pictures or thedifference specified by the information.MVF=MV×TRF/TRD  Equation 1MVB=(TRF−TRD)×MV/TRD  Equation 2where MVF and MVB respectively represent horizontal components andvertical components of the motion vectors, and the plus and minus signsindicate directions of the motion vectors.

By the way, as for selection of a coding mode, a method for reducingcoding error with a smaller amount of bits is generally selected. Themode selection unit 109 outputs the determined coding mode to the bitstream generation unit 104. If the coding mode determined by the modeselection unit 109 is inter picture prediction coding, the motionvectors used for the inter picture prediction coding is outputted to thebit stream generation unit 104 and further stored in the motion vectorstorage unit 116. When the direct mode is selected, the motion vectorswhich are calculated according to Equation 1 and Equation 2 and used fordirect mode are stored in the motion vector storage unit 116.

The mode selection unit 109 also generates predictive image data basedon the determined coding mode for outputting to the differencecalculation unit 102 and the addition unit 106, although it does notoutput the predictive image data if it selects the intra picture coding.In addition, when selecting the intra picture coding, the mode selectionunit 109 controls the switches 111 and 112 to connect to “a” side and“c” side respectively, and when selecting the inter picture predictioncoding or direct mode, it controls the switches 111 and 112 to connectto “b” side and “d” side respectively. The case will be explained belowwhere the mode selection unit 109 selects the inter picture predictioncoding or the direct mode.

The difference calculation unit 102 receives the image data of themacroblock of the picture B7 read out from the reordering memory 101 andthe predictive image data outputted from the mode selection unit 109.The difference calculation unit 102 calculates the difference betweenthe image data of the macroblock of the picture B7 and the predictiveimage data, and generates the residual error image data for outputtingto the residual error coding unit 103.

The residual error coding unit 103 performs coding processing such asfrequency transform and quantization on the inputted residual errorimage data and thus generates the coded data for outputting to the bitstream generation unit 104 and the residual error decoding unit 105.

The bit stream generation unit 104 performs variable length coding orthe like on the inputted coded data, and further adds information suchas motion vectors and a coding mode and so on to that data forgenerating and outputting a bit stream.

On the other hand, the residual error decoding unit 105 performsdecoding processing such as inverse quantization and inverse frequencytransform on the inputted coded data and generates the decodeddifferential image data for outputting to the addition unit 106. Theaddition unit 106 adds the decoded differential image data and thepredictive image data inputted by the mode selection unit 109 forgenerating the decoded image data, and stores it in the referencepicture memory 107.

That is the completion of coding one macroblock in the picture B7.According to the same processing, the remaining macroblocks in thepicture B7 are coded. And after all the macroblocks of the picture B7are coded, the picture B6 is coded.

(Coding of Picture B6)

Since the picture B6 is a B-picture, B6 is coded using inter pictureprediction with reference to two previously processed pictures locatedearlier or later than B6 in display order. The B-picture B6 refers tothe picture P5 as a forward reference picture and the picture B7 as abackward reference picture, as described above. Since the picture B6 isnot used as a reference picture for coding other pictures, the codingcontrol unit 110 controls the switch 113 to be ON and the switches 114and 115 to be OFF, which causes the macroblock of the picture B6 readout from the reordering memory 101 to be inputted to the motion vectorestimation unit 108, the mode selection unit 109 and the differencecalculation unit 102.

Using the decoded picture data of the picture P5 and the decoded picturedata of the picture B7 which are stored in the reference picture memory107 as a forward reference picture and a backward reference picturerespectively, the motion vector estimation unit 108 estimates theforward motion vector and the backward motion vector for the macroblockin the picture B6. And the motion vector estimation unit 108 outputs theestimated motion vectors to the mode selection unit 109.

The mode selection unit 109 determines the coding mode for themacroblock in the picture B6 using the motion vectors estimated by themotion vector estimation unit 108.

Here, the first example of direct mode coding operation for themacroblock in the picture B6 will be explained with reference to FIG.7B. FIG. 7B is an illustration showing motion vectors in direct mode,and specifically showing the case where the block a in the picture B6 iscoded in direct mode. In this case, a motion vector c, which has beenused for coding a block b in the picture B7 is utilized. The block b isco-located with the block a, and the picture B7 is a backward referencepicture of the picture B6. Here, it is assumed that the block b is codedby forward reference only or bi-predictive reference and the forwardmotion vector of the block b is the motion vector c. It is also assumedthat the motion vector c is stored in the motion vector storage unit116. The block a is bi-predicted from the forward reference picture P5and the backward reference picture B7 using motion vectors generatedutilizing the motion vector c. For example, if a method of generatingmotion vectors parallel to the motion vector c is used, as is the caseof the above-mentioned picture B7, the motion vector d and the motionvector e are used for the picture P5 and the picture B7 respectively forcoding the block a.

In this case where the forward motion vector d is MVF, the backwardmotion vector e is MVB, the motion vector c is MV, the temporal distancebetween the backward reference picture B7 for the current picture B6 andthe picture P5 which the block b in the backward reference picture B7refers to is TRD, and the temporal distance between the current pictureB6 and the forward reference picture P5 is TRF respectively, the motionvector d MVF and the motion vector e MVB are respectively calculated byabove-mentioned Equation 1 and Equation 2. Note that the temporaldistance between the pictures can be determined based on the informationindicating display order of the pictures or the difference specified bythe information, for instance.

As described above, in direct mode, by scaling the forward motion vectorof a backward reference B-picture, there is no need to transmit motionvector information, and motion prediction efficiency can be improved.Accordingly, coding efficiency can be improved. In addition, by usingreference pictures temporally closest available in display order as aforward reference picture and a backward reference picture, codingefficiency can be increased.

Next, the second example of the direct mode will be explained withreference to FIG. 7B. In this case, the motion vector, which has beenused for coding the block b in the picture B7, is utilized. The block bis co-located with the block a, and the picture B7 is a backwardreference picture for the picture B6. Here, it is assumed that the blockb has been coded in direct mode and the forward motion vector which hasbeen substantially used for coding the block b is the motion vector c.Specifically, the motion vector c is obtained by scaling the motionvector used for coding a block i, co-located with the block b, in thepicture P9 that is the backward reference picture for the picture B7.The motion vector c stored in the motion vector storage unit 116 isused, or the motion vector c is obtained by reading out from the motionvector storage unit 116 the motion vector of the block i in the pictureP9 which has been used for coding the block b in direct mode andcalculating based on that motion vector. When the motion vector which isobtained by scaling for coding the block b in the picture B7 in directmode is stored in the motion vector storage unit 116, only the forwardmotion vector needs to be stored. The block a is bi-predicted from theforward reference picture P5 and the backward reference picture B7 usingthe motion vectors generated utilizing the motion vector c. For example,if a method of generating motion vectors parallel to the motion vector cis used, as is the case of the above-mentioned first example, motionvectors used for coding the block a are the motion vector d and themotion vector e for the picture P5 and the picture B7 respectively.

In this case, the forward motion vector d MVF and the backward motionvector e MVB of the block a are respectively calculated byabove-mentioned Equation 1 and Equation 2, as in the case of the firstexample.

As described above, in direct mode, since the forward motion vector of abackward reference B-picture which has been substantially used forcoding the B-picture in direct mode is scaled, there is no need totransmit the motion vector information, and motion prediction efficiencycan be improved even if the co-located block in the backward referencepicture has been coded in direct mode. Accordingly, coding efficiencycan be improved. In addition, by using reference pictures which aretemporally closest available in display order as a forward referencepicture and a backward reference picture, coding efficiency can beincreased.

Next, the third example of direct mode will be explained with referenceto FIG. 7C. FIG. 7C is an illustration showing motion vectors in directmode, and specifically showing the case where the block a in the pictureB6 is coded in direct mode. In this case, the motion vector which hasbeen used for coding the block b in the picture B7 is utilized. Thepicture B7 is a backward reference picture for the picture B6, and theblock b in the picture B7 is co-located with the block a in the pictureB6. Here, it is assumed that the block b has been coded using a backwardmotion vector only and the backward motion vector used for coding theblock b is a motion vector f. Specifically, the motion vector f isassumed to be stored in the motion vector storage unit 116. The block ais bi-predicted from the forward reference picture P5 and the backwardreference picture B7 using motion vectors generated utilizing the motionvector f. For example, if a method of generating motion vectors parallelto the motion vector f is used, as is the case of the above-mentionedfirst example, motion vectors used for coding the block a are the motionvector g and the motion vector h for the picture P5 and the picture B7respectively.

In this case, where the forward motion vector g is MVF, the backwardmotion vector h is MVB, the motion vector f is MV, the temporal distancebetween the backward reference picture B7 for the current picture B6 andthe picture P9 which the block in the backward reference picture B7 isTRD, the temporal distance between the current picture B6 and theforward reference picture P5 is TRF, and the temporal distance betweenthe current picture B6 and the backward reference picture B7 is TRBrespectively, the motion vector g MVF and the motion vector h MVB arerespectively calculated by Equation 3 and Equation 4.MVF=−TRF×MV/TRD  Equation 3MVB=TRB×MV/TRD  Equation 4

As described above, in direct mode, since the backward motion vector ofa co-located block in a backward reference B-picture which has been usedfor coding the block is scaled, there is no need to transmit motionvector information, and motion prediction efficiency can be improvedeven if the co-located block in the backward reference picture has onlythe backward motion vector. Accordingly, the coding efficiency can beimproved. In addition, by using reference pictures which are temporallyclosest available in display order as a forward reference picture and abackward reference picture, coding efficiency can be increased.

Next, the fourth example of direct mode will be explained with referenceto FIG. 7D. FIG. 7D is an illustration showing motion vectors in directmode, and specifically showing the case where the block a in the pictureB6 is coded in direct mode. In this case, the motion vector which hasbeen used for coding the block b in the picture B7 is utilized. Thepicture B7 is the backward reference picture for the picture B6, and theblock b is co-located with the block a in the picture B6. Here, it isassumed that the block b has been coded using the backward motion vectoronly, as is the case of the third example, and the backward motionvector used for coding the block b is the motion vector f. Specifically,the motion vector f is assumed to be stored in the motion vector storageunit 116. The block a is bi-predicted from the reference picture P9which is referred to by the motion vector f and the backward referencepicture B7 using motion vectors generated utilizing the motion vector f.For example if a method of generating motion vectors parallel to themotion vector f is used, as is the case of the above-mentioned firstexample, motion vectors used for coding the block a are the motionvector 9 and the motion vector h for the picture P9 and the picture B7respectively.

In this case, where the forward motion vector g is MVF, the backwardmotion vector h is MVB, the motion vector f is MV, the temporal distancebetween the backward reference picture B7 for the current picture B6 andthe picture P9 which the block in the backward reference picture B7refers to is TRD, and the temporal distance between the current pictureB6 and the picture P9 which the block b in the backward referencepicture B7 refers to is TRF respectively, the motion vector g MVF andthe motion vector h MVB are respectively calculated by Equation 1 andEquation 2.

As described above, in direct mode, by scaling the backward motionvector of a co-located block in a backward reference B-picture which hasbeen used for coding the block, there is no need to transmit motionvector information, and motion prediction efficiency can be improvedeven if the co-located block in the backward reference picture has onlythe backward motion vector. Accordingly, coding efficiency can beimproved. In addition, by using a picture referred to by the backwardmotion vector as a forward reference picture, and a reference picturewhich is temporally closest available in display order as a backwardreference picture, coding efficiency can be increased.

Next, the fifth example of the direct mode will be explained withreference to FIG. 8A. FIG. 8A is an illustration showing motion vectorsin direct mode, and specifically showing the case where the block a ofthe picture B6 is coded in direct mode. In this case, on the assumptionthat the value of the motion vectors is “0”, bi-predictive reference isperformed for motion compensation, using the picture P5 as a forwardreference picture and the picture B7 as a backward reference picture.

As mentioned above, by forcing the motion vector “0” in direct mode,when the direct mode is selected, there is no need to transmit themotion vector information nor to scale the motion vector, and thus theprocessing volume can be reduced.

Next, the sixth example of the direct mode will be explained withreference to FIG. 8B. FIG. 8B is an illustration showing motion vectorsin direct mode, and specifically showing the case where the block a inthe picture B6 is coded in direct mode. In this case, the motion vectorg which has been used for coding the block f in the picture P9 isutilized. The picture P9 is located later than the picture B6, and theblock f is co-located with the block a in the picture B6. The motionvector g is stored in the motion vector storage unit 116. The block a isbi-predicted from the forward reference picture P5 and the backwardreference picture B7 using motion vectors generated utilizing the motionvector g. For example, if a method of generating motion vectors parallelto the motion vector g is used, as is the case of the above-mentionedfirst example, motion vectors used for coding the block a are the motionvector h and the motion vector i for the picture P5 and the picture B7respectively for coding the block a.

In this case, where the forward motion vector h is MVF, the backwardmotion vector i is MVB, the motion vector g is MV, the temporal distancebetween the picture P9 which is located later in display order than thecurrent picture B6 and the picture P5 which the block f in the pictureP9 refers to is TRD, the temporal distance between the current pictureB6 and the forward reference picture P5 is TRF, and the temporaldistance between the current picture B6 and the backward referencepicture B7 is TRB respectively, the motion vector h MVF and the motionvector i MVB are respectively calculated by Equation 1 and Equation 5.MVB=−TRB×MV/TRD  Equation 5

As described above, in direct mode, by scaling the motion vector of theP-picture which is located later in display order, there is no need tostore the motion vector of a B-picture if the B-picture is the backwardreference picture, and there is also no need to transmit the motionvector information. In addition, by using reference pictures which aretemporally closest in display order as a forward reference picture and abackward reference picture, coding efficiency can be increased.

Next, the seventh example of the direct mode will be explained withreference to FIG. 8C. FIG. 8C is an illustration showing motion vectorsin direct mode, and specifically showing the case where the block a inthe picture B6 is coded in direct mode. This example shows the casewhere the above-mentioned assignment of relative indices to the picturenumbers is changed (remapped) and the picture P9 is a backward referencepicture. In this case, the motion vector g which has been used forcoding the block f in the picture P9 is utilized. The picture P9 is thebackward reference picture for the picture B7, and the block f isco-located with the block a in the picture B6. The motion vector g isstored in the motion vector storage unit 116. The block a isbi-predicted from the forward reference picture P5 and the backwardreference picture P9 using motion vectors generated utilizing the motionvector g. For example, if a method of generating motion vectors parallelto the motion vector g, as is the case of the above-mentioned firstexample, motion vectors used for coding the block a are the motionvector h and the motion vector i for the picture P5 and the picture P9respectively.

In this case, where the forward motion vector h is MVF, the backwardmotion vector i is MVB, the motion vector g is MV, the temporal distancebetween the backward reference picture P9 for the current picture B6 andthe picture P5 which the block f in the picture P9 refers to is TRD, andthe temporal distance between the current picture B6 and the forwardreference picture P5 is TRF respectively, the motion vector h MVF andthe motion vector i MVB are respectively calculated by Equation 1 andEquation 2.

As described above, in direct mode, the motion vector of the previouslycoded picture can be scaled even if the relative indices to the picturenumbers are remapped, and when the direct mode is selected, there is noneed to transmit the motion vector information.

When the block a in the picture B6 is coded in direct mode, the block inthe backward reference picture for the picture B6 which is co-locatedwith the block a is coded by the forward reference only, bi-predictivereference, or direct mode. And when a forward motion vector has beenused for this coding, this forward motion vector is scaled, and theblock a is coded in direct mode, as is the case of the above-mentionedfirst, second or seventh example. On the other hand, when the blockco-located with the block a has been coded by backward reference onlyusing a backward motion vector, this backward motion vector is scaled,and the block a is coded in direct mode, as is the case of theabove-mentioned third or fourth example.

Above-mentioned direct mode is applicable not only to the case where atime interval between pictures is fixed but also to the case where it isvariable.

The mode selection unit 109 outputs the determined coding mode to thebit stream generation unit 104. Also, the mode selection unit 109generates predictive image data based on the determined coding mode andoutputs it to the difference calculation unit 102. However, if selectingintra picture coding, the mode selection unit 109 does not outputpredictive image data. The mode selection unit 109 controls the switches111 and 112 so as to be connected to “a” side and “c” side respectivelyif selecting intra picture coding, and controls the switches 111 and 112so as to be connected to “b” side and “d” side if selecting interpicture prediction coding or a direct mode. If the determined codingmode is inter picture prediction coding, the mode selection unit 109outputs the motion vectors used for the inter picture prediction codingto the bit stream generation unit 104. Since the picture B6 is not usedas a reference picture for coding other pictures, there is no need tostore the motion vectors used for the inter picture prediction coding inthe motion vector storage unit 116. The case will be explained belowwhere the mode selection unit 109 selects the inter picture predictioncoding or the direct mode.

The difference calculation unit 102 receives the image data of themacroblock in the picture B6 read out from the reordering memory 101 andthe predictive image data outputted from the mode selection unit 109.The difference calculation unit 102 calculates the difference betweenthe image data of the macroblock in the picture B6 and the predictiveimage data and generates the residual error image data for outputting tothe residual error coding unit 103. The residual error coding unit 103performs coding processing such as frequency transform and quantizationon the inputted residual error image data, and thus generates the codeddata for outputting to the bit stream generation unit 104.

The bit stream generation unit 104 performs variable length coding orthe like on the inputted coded data, further adds information such asmotion vectors and a coding mode and so on to the data, and generatesthe bit stream for outputting.

That is the completion of coding one macroblock in the picture B6.According to the same processing, the remaining macroblocks in thepicture B6 are coded. And after all the macroblocks in the picture B6are coded, the picture B8 is coded.

(Coding of Picture B8)

Since a picture B8 is a B-picture, inter picture prediction coding isperformed for the picture B8 with reference to two previously processedpictures located earlier or later than B6 in display order. TheB-picture B8 refers to the picture B7 as a forward reference picture andthe picture P9 as a backward reference picture, as described above.Since the picture B8 is not used as a reference picture for coding otherpictures, the coding control unit 110 controls the switch 113 to be ONand the switches 114 and 115 to be OFF, which causes the macroblocks inthe picture B8 read out from the reordering memory 101 to be inputted tothe motion vector estimation unit 108, the mode selection unit 109 andthe difference calculation unit 102.

Using the decoded picture data of the picture B7 and the decoded picturedata of the picture P9 which are stored in the reference picture memory107 as a forward reference picture and a backward reference picturerespectively, the motion vector estimation unit 108 estimates theforward motion vector and the backward motion vector for the macroblockin the picture B8. And the motion vector estimation unit 108 outputs theestimated motion vectors to the mode selection unit 109.

The mode selection unit 109 determines the coding mode for themacroblock in the picture B8 using the motion vectors estimated by themotion vector estimation unit 108.

Here, the case where the macroblock in the picture B8 is coded using thedirect mode will be explained with reference to FIG. 8D. FIG. 8D is anillustration showing motion vectors in direct mode, and specificallyshowing the case where a block a in the picture B8 is coded in directmode. In this case, a motion vector c which has been used for coding ablock b in the backward picture P9 is utilized. The reference picture P9is located later than the picture B8, and the block b in the picture P9is co-located with the block a. Here, it is assumed that the block b hasbeen coded by forward reference and the forward motion vector for theblock b is the motion vector c. The motion vector c is stored in themotion vector storage unit 116. The block a is bi-predicted from theforward reference picture B7 and the backward reference picture P9 usingmotion vectors generated utilizing the motion vector c. For example, ifa method of generating motion vectors parallel to the motion vector c isused, as is the case of the above-mentioned picture B7, the motionvector d and the motion vector e are used for the picture B7 and thepicture P9 respectively for coding the block a.

In this case where the forward motion vector d is MVF, the backwardmotion vector e is MVB, the motion vector c is MV, the temporal distancebetween the backward reference picture P9 for the current picture B8 andthe picture P5 which the block b in the backward reference picture P9refers to is TRD, the temporal distance between the current picture B8and the forward reference picture B7 is TRF, and the temporal distancebetween the current picture B8 and the backward reference picture P9 isTRB respectively, the motion vector d MVF and the motion vector e MVBare respectively calculated by Equation 1 and Equation 5.

As described above, in direct mode, by scaling the forward motion vectorof the backward reference picture, when the direct mode is selected,there is no need to transmit the motion vector information and themotion prediction efficiency can be improved. Accordingly, codingefficiency can be improved. In addition, by using reference pictureswhich are temporally closest available in display order as forward andbackward reference pictures, coding efficiency can be increased.

Above-mentioned direct mode is applicable not only to the case where atime interval between pictures is fixed but also to the case where it isvariable.

The mode selection unit 109 outputs the determined coding mode to thebit stream generation unit 104. Also, the mode selection unit 109generates predictive image data based on the determined coding mode andoutputs it to the difference calculation unit 102. However, if selectingintra picture coding, the mode selection unit 109 does not outputpredictive image data. The mode selection unit 109 controls the switches111 and 112 so as to be connected to “a” side and “c” side respectivelyif selecting intra picture coding, and controls the switches 111 and 112so as to be connected to “b” side and “d” side if selecting interpicture prediction coding or direct mode. If the determined coding modeis inter picture prediction coding, the mode selection unit 109 outputsthe motion vectors used for the inter picture prediction coding to thebit stream generation unit 104. Since the picture B8 is not be used as areference picture for coding other pictures, there is no need to storethe motion vectors used for the inter picture prediction coding in themotion vector storage unit 116. The case will be explained below wherethe mode selection unit 109 selects the inter picture prediction codingor direct mode.

The difference calculation unit 102 receives the image data of themacroblock in the picture B8 read out from the reordering memory 101 andthe predictive image data outputted from the mode selection unit 109.The difference calculation unit 102 calculates the difference betweenthe image data of the macroblock in the picture B8 and the predictiveimage data and generates the residual error image data for outputting tothe residual error coding unit 103. The residual error coding unit 103performs coding processing such as frequency transform and quantizationon the inputted residual error image data and thus generates the codeddata for outputting to the bit stream generation unit 104.

The bit stream generation unit 104 performs variable length coding orthe like on the inputted coded data, further adds information such asmotion vectors and a coding mode and so on to the data, and generatesthe bit stream for outputting.

That is the completion of coding one macroblock in the picture B8.According to the same processing, the remaining macroblocks in thepicture B8 are coded.

According to the above-mentioned respective coding procedures for thepictures P9, B7, B6 and B8, other pictures are coded depending on theirpicture types and temporal locations in display order.

In the above-mentioned embodiment, the moving picture coding methodaccording to the present invention has been explained taking the casewhere the picture prediction structure as shown in FIG. 6A is used as anexample. FIG. 12 is an illustration showing this picture predictionstructure hierarchically. In FIG. 12, arrows indicate predictionrelations, in which the pictures pointed by the arrows refer to thepictures located at the origins of the arrows. In the picture predictionstructure as shown in FIG. 6A, the coding order is determined by givinga top priority to the pictures which are farthest from the previouslyprocessed pictures in display order, as shown in FIG. 12. For example,the picture farthest from an I-picture or a P-picture is that located inthe center of the consecutive B-pictures. Therefore, if the picture P5and P9 have been coded, the picture B7 is to be coded next. And if thepictures P5, B7 and P9 have been coded, the pictures B6 and B8 are to becoded next.

In addition, the moving picture coding method according to the presentinvention can be used for other picture prediction structures than thoseas shown in FIG. 6 and FIG. 12, so as to produce the effects of thepresent invention. FIGS. 9˜11 show the examples of other pictureprediction structures.

FIG. 9 shows the case where 3 B-pictures are located between I-picturesand P-pictures and the B-picture closest from the previously processedpicture is selected for coding first. FIG. 9A is a diagram showingprediction relations between respective pictures arranged in displayorder, and FIG. 9B is a diagram showing the sequence of picturesreordered in coding order (a bit stream). FIG. 13 is a hierarchicaldiagram of the picture prediction structure corresponding to FIG. 9A. Inthe picture prediction structure as shown in FIG. 9A, the picturesclosest in display order from the previously processed pictures arecoded first, as shown in FIG. 13. For example, if the pictures P5 and P9have been coded, the pictures B6 and B8 are to be coded next. If thepictures P5, B6, B8 and P9 have been coded, the picture B7 is to becoded next.

FIG. 10 shows the case where 5 B-pictures are located between I-picturesand P-pictures and the B-picture which is farthest from the previouslyprocessed picture is selected for coding first. FIG. 10A is a diagramshowing prediction relations between respective pictures arranged indisplay order, and FIG. 10B is a diagram showing the sequence ofpictures reordered in coding order (a bit stream). FIG. 14 is ahierarchical diagram of the picture prediction structure correspondingto FIG. 10A. In the picture prediction structure as shown in FIG. 10A,the coding order is determined by giving a top priority to the picturesfarthest in display order from the previously processed pictures, asshown in FIG. 14. For example, the picture farthest from an I-picture ora P-picture is the B-picture in the center of the consecutiveB-pictures. Therefore, if the pictures P7 and P13 have been coded, thepicture B10 is to be coded next. If the pictures P7, B10 and P13 havebeen coded, the pictures B8, B9, B11 and B12 are to be coded next.

FIG. 11 shows the case where 5 B-pictures are located between I-picturesand P-pictures and the B-picture which is closest from the previouslyprocessed picture is selected for coding first. FIG. 11A is a diagramshowing prediction relations between respective pictures arranged indisplay order, and FIG. 11B is a diagram showing the sequence ofpictures reordered in coding order (a bit stream). FIG. 15 is ahierarchical diagram of the picture prediction structure correspondingto FIG. 11A. In the picture prediction structure as shown in FIG. 11A,the pictures closest in display order from the previously processedpictures are coded first, as shown in FIG. 15. For example, if thepictures P5 and P9 have been coded, the pictures B8 and B12 are to becoded next. If the pictures P5, B8, B12 and P9 have been coded, thepictures B9 and B11 are to be coded next. Furthermore, if the picturesP5, B8, B9, B11, B12 and P9 have been coded, the picture B10 is to becoded next.

As described above, according to the moving picture coding method of thepresent invention, when inter picture prediction coding is performed ona plurality of B-pictures located between I-pictures and P-picturesusing bi-predictive reference, they are coded in another order thandisplay order. For that purpose, the pictures located as close to thecurrent picture as possible in display order are used as forward andbackward pictures. As a reference picture, a B-picture is also used ifit is available. When a plurality of B-pictures located betweenI-pictures and P-pictures are coded in different order from displayorder, the picture farthest from the previously processed picture is tobe coded first. Or, when a plurality of B-pictures located betweenI-pictures and P-pictures are coded in different order from displayorder, the picture closest from the previously processed picture is tobe coded first.

According to the moving picture coding method of the present invention,above-mentioned operation enables to use a picture closer to a currentB-picture in display order as a reference picture for coding it.Prediction efficiency is thus increased for motion compensation andcoding efficiency is increased.

In addition, according to the moving picture coding method of thepresent invention, for coding a block in a B-picture in direct mode withreference to a B-picture previously coded as a backward referencepicture, if the co-located block in the backward reference B-picture hasbeen coded by forward reference or bi-predictive reference, a motionvector obtained by scaling the forward motion vector of the backwardreference B-picture is used as a motion vector in direct mode.

As mentioned above, in direct mode, by scaling a forward motion vectorof a backward reference B-picture, there is no need to transmit motionvector information, and prediction efficiency can be increased. Inaddition, by using a reference picture temporally closest in displayorder as a forward reference picture, coding efficiency can beincreased.

Or, if a co-located block in a backward reference B-picture is coded indirect mode, a motion vector obtained by scaling the forward motionvector substantially used in direct mode is used as a motion vector indirect mode.

As mentioned above, in direct mode, by scaling a forward motion vectorof a backward reference B-picture which has been substantially used forthe direct mode coding, there is no need to transmit motion vectorinformation, and prediction efficiency can be increased even if theco-located block in the backward reference picture is coded in directmode. In addition, coding efficiency can be improved by using atemporally closest reference picture as a forward reference picture.

Or, if a co-located block in a backward reference B-picture is coded bybackward reference, motion vectors obtained by scaling the backwardmotion vector of the block is used as motion vectors in direct mode.

As mentioned above, in direct mode, by scaling a backward motion vectorwhich has been used for coding a co-located block in the backwardreference B-picture, there is no need to transmit motion vectorinformation, and prediction efficiency can be increased even if theco-located block in the backward reference picture has only a backwardmotion vector. In addition, by using a temporally closest referencepicture as a forward reference picture, coding efficiency can beimproved.

Or, if a co-located block in a backward reference B-picture is coded bybackward reference, motion vectors obtained by scaling the backwardmotion vector used for that coding, with reference to the picturereferred to by this backward motion vector and the backward referencepicture, are used as motion vectors in direct mode.

As mentioned above, in direct mode, by scaling a backward motion vectorwhich has been used for coding a co-located block in the backwardreference B-picture, there is no need to transmit motion vectorinformation, and prediction efficiency can be increased even if theco-located block in the backward reference picture has only a backwardmotion vector. Accordingly, coding efficiency can be improved. Inaddition, by using a picture referred to by the backward motion vectoras a forward reference picture and a reference picture temporallyclosest available in display order as a backward reference picture,coding efficiency can be increased.

Or, in direct mode, a motion vector which is forced to be set to “0” isused.

By forcing a motion vector to be set to “0” in direct mode, when thedirect mode is selected, there is no need to transmit the motion vectorinformation nor to scale the motion vector, and therefore the processingvolume can be reduced.

In addition, according to the moving picture coding method of thepresent invention, for coding a block in a B-picture in direct mode withreference to a B-picture which has been previously coded as a backwardreference picture, a motion vector obtained by scaling the forwardmotion vector which has been used for coding the co-located block in thelater P-picture is used as a motion vector in direct mode.

As mentioned above, in direct mode, by scaling a motion vector of alater P-picture, if the backward reference picture is a B-picture, thereis no need to store the motion vectors of the B-picture and there is noneed to transmit the motion vector information, and thus predictionefficiency can be increased. In addition, by using a temporally closestreference picture as a forward reference picture, coding efficiency canbe improved.

When assignment of relative indices to picture numbers is changed and aco-located block in a backward reference picture has been coded byforward reference, motion vectors obtained by scaling that forwardmotion vector are used as motion vectors in direct mode.

As mentioned above, in direct mode, a motion vector of a previouslycoded picture can be scaled even if assignment of relative indices topicture numbers is changed, and there is no need to transmit motionvector information.

In the present embodiment, the case has been explained where motioncompensation is made in every 16 (horizontal)×16 (vertical) pixels andresidual error image data is coded in every 8 (horizontal)×8 (vertical)pixels or 4 (horizontal)×4 (vertical) pixels, but other size (number ofpixels included) may be applied.

Also, in the present embodiment, the case has been explained whereconsecutive 3 or 5 B-pictures are located, but other number of picturesmay be located.

Further, in the present embodiment, the case has been explained whereone of intra picture coding, inter picture prediction coding usingmotion vectors and inter picture prediction coding without using motionvectors is selected as a coding mode for P-pictures, and one of intrapicture coding, inter picture prediction coding using a forward motionvector, inter picture prediction coding using a backward motion vector,inter picture prediction coding using a bi-predictive motion vectors anddirect mode is selected for B-pictures, but other coding mode may beused.

Also, in the present embodiment, seven examples of direct mode have beenexplained, but a method which is uniquely determined in every macroblockor block may be used, or any of a plurality of methods in everymacroblock or block may be selected. If a plurality of methods are used,information indicating which type of direct mode has been used isdescribed in a bit stream.

In addition, in the present embodiment, the case has been explainedwhere a P-picture is coded with reference to one previously coded I orP-picture which is located temporally earlier or later in display orderthan the current P-picture, and a B-picture is coded with reference totwo previously processed neighboring pictures which are located earlieror later in display order than the current B-picture, respectively.However, in the case of a P-picture, the P-picture may be coded withreference to at most one picture for each block from among a pluralityof previously coded I or P pictures as candidate reference pictures, andin the case of a B-picture, the B-picture may be coded with reference toat most two pictures for each block from among a plurality of previouslycoded neighboring pictures which are located temporally earlier or laterin display order as candidate reference pictures.

In addition, when storing motion vectors in the motion vector storageunit 116, the mode selection unit 109 may store both forward andbackward motion vectors or only a forward motion vector, if a currentblock is coded by bi-predictive reference or in direct mode. If itstores only the forward motion vector, the volume stored in the motionvector storage unit 116 can be reduced.

Second Embodiment

FIG. 16 is a block diagram showing a structure of a moving picturedecoding apparatus using a moving picture decoding method according toan embodiment of the present invention.

As shown in FIG. 16, the moving picture decoding apparatus includes abit stream analysis unit 1401, a residual error decoding unit 1402, amode decoding unit 1403, a frame memory control unit 1404, a motioncompensation decoding unit 1405, a motion vector storage unit 1406, aframe memory 1407, an addition unit 1408 and switches 1409 and 1410.

The bit stream analysis unit 1401 extracts various types of data such ascoding mode information and motion vector information from the inputtedbit stream. The residual error decoding unit 1402 decodes the residualerror coded data inputted from the bit stream analysis unit 1401 andgenerates residual error image data. The mode decoding unit 1403controls the switches 1409 and 1410 with reference to the coding modeinformation extracted from the bit stream.

The frame memory control unit 1404 outputs the decoded picture datastored in the frame memory 1407 as output pictures based on theinformation indicating the display order of the pictures inputted fromthe bit stream analysis unit 1401.

The motion compensation decoding unit 1405 decodes the information ofthe reference picture numbers and the motion vectors, and obtains motioncompensation image data from the frame memory 1407 based on the decodedreference picture numbers and motion vectors. The motion vector storageunit 1406 stores motion vectors.

The addition unit 1408 adds the residual error coded data inputted fromthe residual error decoding unit 1402 and the motion compensation imagedata inputted from the motion compensation decoding unit 1405 forgenerating the decoded image data. The frame memory 1407 stores thegenerated decoded image data.

Next, the operation of the moving picture decoding apparatus asstructured as above will be explained. Here, it is assumed that the bitstream generated by the moving picture coding apparatus is inputted tothe moving picture decoding apparatus. Specifically, it is assumed thata P-picture refers to one previously processed neighboring I orP-picture which is located earlier or later than the current P-picturein display order, and a B-picture refers to two previously codedneighboring pictures which are located earlier or later than the currentB-picture in display order.

In this case, the pictures in the bit stream are arranged in the orderas shown in FIG. 6B. Decoding processing of pictures P9, B7, B6 and B8will be explained below in this order.

(Decoding of Picture P9)

The bit stream of the picture P9 is inputted to the bit stream analysisunit 1401. The bit stream analysis unit 1401 extracts various types ofdata from the inputted bit stream. Here, various types of data mean modeselection information, motion vector information and others. Theextracted mode selection information is outputted to the mode decodingunit 1403. The extracted motion vector information is outputted to themotion compensation decoding unit 1405. And the residual error codeddata is outputted to the residual error decoding unit 1402.

The mode decoding unit 1403 controls the switches 1409 and 1410 withreference to the coding mode selection information extracted from thebit stream. If intra picture coding is selected as a coding mode, themode decoding unit 1403 controls the switches 1409 and 1410 so as to beconnected to “a” side and “c” side respectively. If inter pictureprediction coding is selected as a coding mode, the mode decoding unit1403 controls the switches 1409 and 1410 so as to be connected to “b”side and “d” side respectively.

The mode decoding unit 1403 also outputs the coding mode selectioninformation to the motion compensation decoding unit 1405. The casewhere the inter picture prediction coding is selected as a coding modewill be explained below. The residual error decoding unit 1402 decodesthe inputted residual error coded data to generate residual error imagedata. The residual error decoding unit 1402 outputs the generatedresidual error image data to the switch 1409. Since the switch 1409 isconnected to “b” side, the residual error image data is outputted to theaddition unit 1408.

The motion compensation decoding unit 1405 obtains motion compensationimage data from the frame memory 1407 based on the inputted motionvector information and the like. The picture P9 has been coded withreference to the picture P5, and the picture P5 has been already decodedand stored in the frame memory 1407. So, the motion compensationdecoding unit 1405 obtains the motion compensation image data from thepicture data of the picture P5 stored in the frame memory 1407, based onthe motion vector information. The motion compensation image datagenerated in this manner is outputted to the addition unit 1408.

When decoding P-pictures, the motion compensation decoding unit 1405stores the motion vector information in the motion vector storage unit1406.

The addition unit 1408 adds the inputted residual error image data andmotion compensation image data to generate decoded image data. Thegenerated decoded image data is outputted to the frame memory 1407 viathe switch 1410.

That is the completion of decoding one macroblock in the picture P9.According to the same processing, the remaining macroblocks in thepicture P9 are decoded in sequence. And after all the macroblocks in thepicture P9 are decoded, the picture B7 is decoded.

(Decoding of Picture B7)

Since the operations of the bit stream analysis unit 1401, the modedecoding unit 1403 and the residual error decoding unit 1402 untilgeneration of residual error image data are same as those for decodingthe picture P9, the explanation thereof will be omitted.

The motion compensation decoding unit 1405 generates motion compensationimage data based on the inputted motion vector information and the like.The picture B7 is coded with reference to the picture P5 as a forwardreference picture and the picture P9 as a backward reference picture,and these pictures P5 and P9 have already been decoded and stored in theframe memory 1407.

If inter picture bi-prediction coding is selected as a coding mode, themotion compensation decoding unit 1405 obtains the forward referencepicture data from the frame memory 1407 based on the forward motionvector information. It also obtains the backward reference picture datafrom the frame memory 1407 based on the backward motion vectorinformation. Then, the motion compensation decoding unit 1405 averagesthe forward and backward reference picture data to generate motioncompensation image data.

When direct mode is selected as a coding mode, the motion compensationdecoding unit 1405 obtains the motion vector of the picture P9 stored inthe motion vector storage unit 1406. Using this motion vector, themotion compensation decoding unit 1405 obtains the forward and backwardreference picture data from the frame memory 1407. Then, the motioncompensation decoding unit 1405 averages the forward and backwardreference picture data to generate motion compensation image data.

The case where the direct mode is selected as a coding mode will beexplained with reference to FIG. 7A again. Here, it is assumed that theblock a in the picture B7 is to be decoded and the block b in thepicture P9 is co-located with the block a. The motion vector of theblock b is the motion vector c, which refers to the picture P5. In thiscase, the motion vector d which is obtained utilizing the motion vectorc and refers to the picture P5 is used as a forward motion vector, andthe motion vector e which is obtained utilizing the motion vector c andrefers to the picture P9 is used as a backward motion vector. Forexample, as a method of utilizing the motion vector c, there is a methodof generating motion vectors parallel to the motion vector c. The motioncompensation image data is obtained by averaging the forward andbackward reference data obtained based on these motion vectors.

In this case where the forward motion vector d is MVF, the backwardmotion vector e is MVB, the motion vector c is MV, the temporal distancebetween the backward reference picture P9 for the current picture B7 andthe picture P5 which the block b in the backward reference picture P9refers to is TRD, and the temporal distance between the current pictureB7 and the forward reference picture P5 is TRF respectively, the motionvector d MVF and the motion vector e MVB are respectively calculated byEquation 1 and Equation 2, where MVF and MVB represent horizontal andvertical components of the motion vectors respectively. Note that thetemporal distance between the pictures can be determined based on theinformation indicating the display order (position) given to respectivepictures or the difference specified by the information.

The motion compensation image data generated in this manner is outputtedto the addition unit 1408. The motion compensation decoding unit 1405stores the motion vector information in the motion vector storage unit1406.

The addition unit 1408 adds the inputted residual error image data andthe motion compensation image data to generate decoded image data. Thegenerated decoded image data is outputted to the frame memory 1407 viathe switch 1410.

That is the completion of decoding one macroblock in the picture B7.According to the same processing, the remaining macroblocks in thepicture B7 are decoded in sequence. And after all the macroblocks of thepicture B7 are decoded, the picture B6 is decoded.

(Decoding of Picture B6)

Since the operations of the bit stream analysis unit 1401, the modedecoding unit 1403 and the residual error decoding unit 1402 untilgeneration of residual error image data are same as those for decodingthe picture P9, the explanation thereof will be omitted.

The motion compensation decoding unit 1405 generates motion compensationimage data based on the inputted motion vector information and the like.The picture B6 has been coded with reference to the picture P5 as aforward reference picture and the picture B7 as a backward referencepicture, and these pictures P5 and B7 have been already decoded andstored in the frame memory 1407.

If inter picture bi-prediction coding is selected as a coding mode, themotion compensation decoding unit 1405 obtains the forward referencepicture data from the frame memory 1407 based on the forward motionvector information. It also obtains the backward reference picture datafrom the frame memory 1407 based on the backward motion vectorinformation. Then, the motion compensation decoding unit 1405 averagesthe forward and backward reference picture data to generate motioncompensation image data.

When the direct mode is selected as a coding mode, the motioncompensation decoding unit 1405 obtains the motion vector of the pictureB7 stored in the motion vector storage unit 1406. Using this motionvector, the motion compensation decoding unit 1405 obtains the forwardand backward reference picture data from the frame memory 1407. Then,the motion compensation decoding unit 1405 averages the forward andbackward reference picture data to generate motion compensation imagedata.

The first example of the case where the direct mode is selected as acoding mode will be explained with reference to FIG. 7B again. Here, itis assumed that the block a in the picture B6 is to be decoded and theblock b in the picture B7 is co-located with the block a. The block bhas been coded by forward reference inter picture prediction orbi-predictive reference inter picture prediction, and the forward motionvector of the block b is the motion vector c, which refers to thepicture P5. In this case, the motion vector d which is obtainedutilizing the motion vector c and refers to the picture P5 is used as aforward motion vector, and the motion vector e which is obtainedutilizing the motion vector c and refers to the picture B7 is used as abackward motion vector. For example, as a method of utilizing the motionvector c, there is a method of generating motion vectors parallel to themotion vector c. The motion compensation image data is obtained byaveraging the forward and backward reference picture data obtained basedon these motion vectors d and e.

In this case where the forward motion vector d is MVF, the backwardmotion vector e is MVB, the motion vector c is MV, the temporal distancebetween the backward reference picture B7 for the current picture B6 andthe picture P5 which the block b in the backward reference picture B7refers to is TRD, and the temporal distance between the current pictureB6 and the forward reference picture P5 is TRF respectively, the motionvector d MVF and the motion vector e MVB are respectively calculated byEquation 1 and Equation 2. Note that the temporal distance betweenpictures may be determined based on the information indicating thedisplay order (position) of the pictures or the difference specified bythe information. Or, as the values of TRD and TRF, predetermined valuesfor respective pictures may be used. These predetermined values may bedescribed in the bit stream as header information.

The second example of the case where the direct mode is selected as acoding mode will be explained with reference to FIG. 7B again.

In this example, the motion vector which has been used for decoding theblock b in the picture B7 is utilized. The picture B7 is the backwardreference picture for the current picture B6, and the block b isco-located with the block a in the picture B6. Here, it is assumed thatthe block b has been coded in direct mode and the motion vector c hasbeen substantially used as a forward motion vector for that coding. Themotion vector c stored in the motion vector storage unit 1406 may beused, or it is calculated by reading out from the motion vector storageunit 1406 the motion vector of the picture P9 which has been used forcoding the block b in direct mode, and then scaling that motion vector.Note that when storing motion vectors in the motion vector storage unit1406, the motion compensation decoding unit 1405 needs to store only theforward motion vector out of the two motion vectors obtained by scalingfor decoding the block b in the picture B7 in direct mode.

In this case, for the block a, the motion vector d which is generatedutilizing the motion vector c and refers to the picture P5 is used as aforward motion vector, and the motion vector e which is generatedutilizing the motion vector c and refers to the picture B7 is used as abackward motion vector. For example, as a method of utilizing the motionvector c, there is a method of generating motion vectors parallel to themotion vector c. The motion compensation image data is obtained byaveraging the forward and backward reference picture data obtained basedon these motion vectors d and e.

In this case, the motion vector d MVF and the motion vector e MVB arerespectively calculated by Equation 1 and Equation 2, as is the case ofthe first example of the direct mode.

Next, the third example of the case where the direct mode is selected asa coding mode will be explained with reference to FIG. 7C again.

In this example, it is assumed that the block a in the picture B6 is tobe decoded, and the block b in the picture B7 is co-located with theblock a. The block b has been coded by backward reference prediction,and the backward motion vector of the block b is a motion vector f,which refers to the picture P9. In this case, for the block a, themotion vector g which is obtained utilizing the motion vector f andrefers to the picture P5 is used as a forward motion vector, and themotion vector h which is obtained utilizing the motion vector f andrefers to the picture B7 is used as a backward motion vector. Forexample, as a method of utilizing the motion vector f, there is a methodof generating motion vectors parallel to the motion vector f. The motioncompensation image data is obtained by averaging the forward andbackward reference picture data obtained based on these motion vectors gand h.

In this case where the forward motion vector g is MVF, the backwardmotion vector h is MVB, the motion vector f is MV, the temporal distancebetween the backward reference picture B7 for the current picture B6 andthe picture P9 which the block b in the backward reference picture B7refers to is TRD, the temporal distance between the current picture B6and the forward reference picture P5 is TRF, and the temporal distancebetween the current picture B6 and the backward reference picture B7 isTRB respectively, the motion vector g MVF and the motion vector h MVBare respectively calculated by Equation 3 and Equation 4.

Next, the fourth example of the case where the direct mode is selectedas a coding mode will be explained with reference to FIG. 7D again.

In this example, it is assumed that the block a in the picture B6 is tobe decoded, and the block b in the picture B7 is co-located with theblock a. The block b has been coded by backward reference prediction asis the case of the third example, and the backward motion vector of theblock b is a motion vector f, which refers to the picture P9. In thiscase, the motion vector g which is obtained utilizing the motion vectorf and refers to the picture P9 is used as a forward motion vector, andthe motion vector h which is obtained utilizing the motion vector f andrefers to the picture B7 is used as a backward motion vector. Forexample, as a method of utilizing the motion vector f, there is a methodof generating motion vectors parallel to the motion vector f. The motioncompensation image data is obtained by averaging the forward andbackward reference picture data obtained based on these motion vectors gand h.

In this case where the forward motion vector g is MVF, the backwardmotion vector h is MVB, the motion vector f is MV, the temporal distancebetween the backward reference picture B7 for the current picture B6 andthe picture P9 which the block b in the backward reference picture B7refers to is TRD, and the temporal distance between the current pictureB6 and the reference picture P9 which the block b in the backwardreference picture B7 refers to is TRF respectively, the motion vector gMVF and the motion vector h MVB are respectively calculated by Equation1 and Equation 2.

Furthermore, the fifth example of the case where the direct mode isselected as a coding mode will be explained with reference to FIG. 8Aagain. Here, it is assumed that a block a in the picture B6 is to bedecoded in direct mode. In this example, the motion vector is set tozero “0”, and motion compensation is performed by bi-predictivereference using the picture P5 as a forward reference picture and thepicture B7 as a backward reference picture.

Next, the sixth example of the case where the direct mode is selected asa coding mode will be explained with reference to FIG. 8B again. Here,it is assumed that a block a in the picture B6 is to be decoded indirect mode. In this example, the motion vector g which has been usedfor decoding the block f in the P-picture P9 is utilized. The picture P9is located later than the current picture B6, and the block f isco-located with the block a. The motion vector g is stored in the motionvector storage unit 1406. The block a is bi-predicted from the forwardreference picture P5 and the backward reference picture B7 using themotion vectors which are obtained utilizing the motion vector g. Forexample, if a method of generating motion vectors parallel to the motionvector g is used, as is the case of the above-mentioned first example,the motion vector h and the motion vector i are used for the picture P5and the picture B7 respectively for obtaining the motion compensationimage data of the block a.

In this case where the forward motion vector h is MVF, the backwardmotion vector i is MVB, the motion vector g is MV, the temporal distancebetween the picture P9 located later than the current picture B6 and thepicture P5 which the block f in the picture P9 refers to is TRD, thetemporal distance between the current picture B6 and the forwardreference picture P5 is TRF, and the temporal distance between thecurrent picture B6 and the backward reference picture B7 is TRBrespectively, the motion vector h MVF and the motion vector i MVB arerespectively calculated by Equation 1 and Equation 5.

Next, the seventh example of the case where the direct mode is selectedas a coding mode will be explained with reference to FIG. 8C again.Here, it is assumed that a block a in the picture B6 is decoded indirect mode. In this example, the assignment of relative indices to theabove-mentioned picture numbers is changed (remapped) and the picture P9is the backward reference picture. In this case, the motion vector gwhich has been used for coding the block f in the picture P9 isutilized. The picture P9 is the backward reference picture for thepicture B6, and the block f is co-located with the block a in thepicture B6. The motion vector g is stored in the motion vector storageunit 1406. The block a is bi-predicted from the forward referencepicture P5 and the backward reference picture P9 using motion vectorsgenerated utilizing the motion vector g. For example, if a method ofgenerating motion vectors parallel to the motion vector g is used, as isthe case of the above-mentioned first example, the motion vector h andthe motion vector i are used for the picture P5 and the picture P9respectively for obtaining the motion compensation image data of theblock a.

In this case, where the forward motion vector h is MVF, the backwardmotion vector i is MVB, the motion vector g is MV, the temporal distancebetween the backward reference picture P9 for the current picture B6 andthe picture P5 which the block f in the picture P9 refers to is TRD, andthe temporal distance between the current picture B6 and the forwardreference picture P5 is TRF respectively, the motion vector h MVF andthe motion vector i MVB are respectively calculated by Equation 1 andEquation 2.

The motion compensation image data generated as above is outputted tothe addition unit 1408. The addition unit 1408 adds the inputtedresidual error image data and the motion compensation image data togenerate decoded image data. The generated decoded image data isoutputted to the frame memory 1407 via the switch 1410.

That is the completion of decoding one macroblock in the picture B6.According to the same processing, the remaining macroblocks in thepicture B6 are decoded in sequence. And after all the macroblocks in thepicture B6 are decoded, the picture B8 is decoded.

(Decoding of Picture B8)

Since the operations of the bit stream analysis unit 1401, the modedecoding unit 1403 and the residual error decoding unit 1402 untilgeneration of residual error image data are same as those for decodingthe picture P9, the explanation thereof will be omitted.

The motion compensation decoding unit 1405 generates motion compensationimage data based on the inputted motion vector information and the like.The picture B8 has been coded with reference to the picture B7 as aforward reference picture and the picture P9 as a backward referencepicture, and these pictures B7 and P9 have been already decoded andstored in the frame memory 1407.

If inter picture bi-prediction coding is selected as a coding mode, themotion compensation decoding unit 1405 obtains the forward referenceimage data from the frame memory 1407 based on the forward motion vectorinformation. It also obtains the backward reference image data from theframe memory 1407 based on the backward motion vector information. Then,the motion compensation decoding unit 1405 averages the forward andbackward reference image data to generate motion compensation imagedata.

When direct mode is selected as a coding mode, the motion compensationdecoding unit 1405 obtains the motion vector of the picture P9 stored inthe motion vector storage unit 1406. Using this motion vector, themotion compensation decoding unit 1405 obtains the forward and backwardreference image data from the frame memory 1407. Then, the motioncompensation decoding unit 1405 averages the forward and backwardreference picture data to generate motion compensation image data.

The case where the direct mode is selected as a coding mode will beexplained with reference to FIG. 8D again. Here, it is assumed that ablock a in the picture B8 is to be decoded and a block b in the backwardreference picture P9 is co-located with the block a. The forward motionvector of the block b is the motion vector c, which refers to thepicture P5. In this case, the motion vector d which is generatedutilizing the motion vector c and refers to the picture B7 is used as aforward motion vector, and the motion vector e which is generatedutilizing the motion vector c and refers to the picture P9 is used as abackward motion vector. For example, as a method of utilizing the motionvector c, there is a method of generating motion vectors parallel to themotion vector c. The motion compensation image data is obtained byaveraging the forward and backward reference image data obtained basedon these motion vectors d and e.

In this case where the forward motion vector d is MVF, the backwardmotion vector e is MVB, the motion vector c is MV, the temporal distancebetween the backward reference picture P9 for the current picture B8 andthe picture P5 which the block b in the backward reference picture P9refers to is TRD, the temporal distance between the current picture B8and the forward reference picture B7 is TRF, and the temporal distancebetween the current picture B8 and the backward reference picture P9 isTRB respectively, the motion vector d MVF and the motion vector e MVBare respectively calculated by Equation 1 and Equation 5.

The motion compensation image data generated in this manner is outputtedto the addition unit 1408. The addition unit 1408 adds the inputtedresidual error image data and the motion compensation image data togenerate decoded image data. The generated decoded image data isoutputted to the frame memory 1407 via the switch 1410.

That is the completion of decoding one macroblock in the picture B8.According to the same processing, the remaining macroblocks in thepicture B8 are decoded in sequence. The other pictures are decodeddepending on their picture types according to the above-mentioneddecoding procedures.

Next, the frame memory control unit 1404 reorders the picture data ofthe pictures stored in the frame memory 1407 in time order as shown inFIG. 6A for outputting as output pictures.

As described above, according to the moving picture decoding method ofthe present invention, a B-picture which has been coded by inter picturebi-prediction is decoded using previously decoded pictures which arelocated close in display order as forward and backward referencepictures.

When the direct mode is selected as a coding mode, reference image datais obtained from previously decoded image data to obtain motioncompensation image data, with reference to a motion vector of apreviously decoded backward reference picture stored in the motionvector storage unit 1406.

According to this operation, when a B-picture has been coded by interpicture bi-prediction using pictures which are located close in displayorder as forward and backward reference pictures, the bit streamgenerated as a result of such coding can be properly decoded.

In the present embodiment, seven examples of the direct mode have beenexplained. However, one method, which is uniquely determined for everymacroblock or block based on the decoding method of a co-located blockin a backward reference picture, may be used, or a plurality ofdifferent methods may be used for every macroblock or block by switchingthem. When a plurality of methods are used, the macroblock or the blockis decoded using information described in a bit stream, indicating whichtype of direct mode has been used. For that purpose, the operation ofthe motion compensation decoding unit 1405 depends upon the information.For example, when this information is added for every block of motioncompensation, the mode decoding unit 1403 determines which type ofdirect mode is used for coding and delivers it to the motioncompensation decoding unit 1405. The motion compensation decoding unit1405 performs decoding processing using the decoding method as explainedin the present embodiment depending upon the delivered type of directmode.

Also, in the present embodiment, the picture structure where threeB-pictures are located between I-pictures and P-pictures has beenexplained, but any other number, four or five, for instance, ofB-pictures may be located.

In addition, in the present embodiment, the explanation has been made onthe assumption that a P-picture is coded with reference to onepreviously coded I or P-picture which is located earlier or later thanthe current P-picture in display order, a B-picture is coded withreference to two previously coded neighboring pictures which are locatedearlier or later than the current B-picture in display order, and thebit stream generated as a result of this coding is decoded. However, inthe case of a P-picture, the P-picture may be coded with reference to atmost one picture for each block from among a plurality of previouslycoded I or P pictures which are located temporally earlier or later indisplay order as candidate reference pictures, and in the case of aB-picture, the B-picture may be coded with reference to at most twopictures for each block from among a plurality of previously codedneighboring pictures which are located temporally earlier or later indisplay order as candidate reference pictures.

Furthermore, when storing motion vectors in the motion vector storageunit 1406, the motion compensation decoding unit 1405 may store bothforward and backward motion vectors, or store only the forward motionvector, if a current block is coded by bi-predictive reference or indirect mode. If only the forward motion vector is stored, the memoryvolume of the motion vector storage unit 1406 can be reduced.

Third Embodiment

If a program for realizing the structures of the moving picture codingmethod or the moving picture decoding method as shown in the aboveembodiments is recorded on a memory medium such as a flexible disk, itbecomes possible to perform the processing as shown in these embodimentseasily in an independent computer system.

FIG. 17 is an illustration showing the case where the processing isperformed in a computer system using a flexible disk which stores themoving picture coding method or the moving picture decoding method ofthe above embodiments.

FIG. 17B shows a front view and a cross-sectional view of an appearanceof a flexible disk, and the flexible disk itself, and FIG. 17A shows anexample of a physical format of a flexible disk as a recording mediumbody. The flexible disk FD is contained in a case F, and a plurality oftracks Tr are formed concentrically on the surface of the disk in theradius direction from the periphery and each track is divided into 16sectors Se in the angular direction. Therefore, as for the flexible diskstoring the above-mentioned program, the moving picture coding method asthe program is recorded in an area allocated for it on the flexible diskFD.

FIG. 17C shows the structure for recording and reproducing the programon and from the flexible disk FD. When the program is recorded on theflexible disk FD, the moving picture coding method or the moving picturedecoding method as a program is written in the flexible disk from thecomputer system Cs via a flexible disk drive. When the moving picturecoding method is constructed in the computer system by the program onthe flexible disk, the program is read out from the flexible disk driveand transferred to the computer system.

The above explanation is made on the assumption that a recording mediumis a flexible disk, but the same processing can also be performed usingan optical disk. In addition, the recording medium is not limited to aflexible disk and an optical disk, but any other medium such as an ICcard and a ROM cassette capable of recording a program can be used.

Following is the explanation of the applications of the moving picturecoding method and the moving picture decoding method as shown in theabove embodiments, and the system using them.

FIG. 18 is a block diagram showing the overall configuration of acontent supply system ex100 for realizing content distribution service.The area for providing communication service is divided into cells ofdesired size, and base stations ex107˜ex110 which are fixed wirelessstations are placed in respective cells.

In this content supply system ex100, devices such as a computer ex111, aPDA (personal digital assistant) ex112, a camera ex113, a mobile phoneex114 and a camera-equipped mobile phone ex115 are connected to theInternet ex 101 via an Internet service provider ex102, a telephonenetwork ex104 and base stations ex107˜ex110.

However, the content supply system ex100 is not limited to theconfiguration as shown in FIG. 18, and a combination of any of them maybe connected. Also, each device may be connected directly to thetelephone network ex104, not through the base stations ex107˜ex110.

The camera ex113 is a device such as a digital video camera capable ofshooting moving pictures. The mobile phone may be a mobile phone of aPDC (Personal Digital Communications) system, a CDMA (Code DivisionMultiple Access) system, a W-CDMA (Wideband-Code Division MultipleAccess) system or a GSM (Global System for Mobile Communications)system, a PHS (Personal Handyphone system) or the like.

A streaming server ex103 is connected to the camera ex113 via the basestation ex109 and the telephone network ex104, which enables livedistribution or the like using the camera ex113 based on the coded datatransmitted from a user. Either the camera ex113 or the server fortransmitting the data may code the data. Also, the moving picture datashot by a camera ex116 may be transmitted to the streaming server ex103via the computer ex111. The camera ex116 is a device such as a digitalcamera capable of shooting still and moving pictures. Either the cameraex116 or the computer ex111 may code the moving picture data. An LSIex117 included in the computer ex111 or the camera ex116 actuallyperforms coding processing. Software for coding and decoding movingpictures may be integrated into any type of storage medium (such as aCD-ROM, a flexible disk and a hard disk) that is a recording mediumwhich is readable by the computer ex111 or the like. Furthermore, acamera-equipped mobile phone ex115 may transmit the moving picture data.This moving picture data is the data coded by the LSI included in themobile phone ex115.

The content supply system ex100 codes contents (such as a music livevideo) shot by users using the camera ex113, the camera ex116 or thelike in the same manner as the above embodiment and transmits them tothe streaming server ex103, while the streaming server ex103 makesstream distribution of the content data to the clients at their request.The clients include the computer ex111, the PDA ex112, the camera ex113,the mobile phone ex114 and so on capable of decoding the above-mentionedcoded data. In the content supply system ex100, the clients can thusreceive and reproduce the coded data, and further can receive, decodeand reproduce the data in real time so as to realize personalbroadcasting.

When each device in this system performs coding or decoding, the movingpicture coding apparatus or the moving picture decoding apparatus, asshown in the above-mentioned embodiment, can be used.

A mobile phone will be explained as an example of the device.

FIG. 19 is a diagram showing the mobile phone ex115 using the movingpicture coding method and the moving picture decoding method explainedin the above embodiments. The mobile phone ex115 has an antenna ex201for sending and receiving radio waves to and from the base stationex110, a camera unit ex203 such as a CCD camera capable of shootingvideo and still pictures, a display unit ex202 such as a liquid crystaldisplay for displaying the data obtained by decoding video and the likeshot by the camera unit ex203 and received by the antenna ex201, a bodyunit including a set of operation keys ex204, a voice output unit ex208such as a speaker for outputting voices, a voice input unit 205 such asa microphone for inputting voices, a storage medium ex207 for storingcoded or decoded data such as data of moving or still pictures shot bythe camera, text data and data of moving or still pictures of receivede-mails, and a slot unit ex206 for attaching the storage medium ex207 tothe mobile phone ex115. The storage medium ex207 includes a flash memoryelement, a kind of EEPROM (Electrically Erasable and Programmable ReadOnly Memory) that is an electrically erasable and rewritable nonvolatilememory, in a plastic case such as a SD card.

The mobile phone ex115 will be further explained with reference to FIG.20. In the mobile phone ex115, a main control unit ex311 for overallcontrolling the display unit ex202 and the body unit including operationkeys ex204 is connected to a power supply circuit unit ex310, anoperation input control unit ex304, a picture coding unit ex312, acamera interface unit ex303, an LCD (Liquid Crystal Display) controlunit ex302, a picture decoding unit ex309, a multiplex/demultiplex unitex308, a record/reproduce unit ex307, a modem circuit unit ex306 and avoice processing unit ex305 to each other via a synchronous bus ex313.

When a call-end key or a power key is turned ON by a user's operation,the power supply circuit unit ex310 supplies respective units with powerfrom a battery pack so as to activate the camera-equipped digital mobilephone ex115 for making it into a ready state.

In the mobile phone ex115, the voice processing unit ex305 converts thevoice signals received by the voice input unit ex205 in conversationmode into digital voice data under the control of the main control unitex311 including a CPU, ROM and RAM, the modem circuit unit ex306performs spread spectrum processing of the digital voice data, and thesend/receive circuit unit ex301 performs digital-to-analog conversionand frequency transform of the data, so as to transmit it via theantenna ex201. Also, in the mobile phone ex115, after the data receivedby the antenna ex201 in conversation mode is amplified and performed offrequency transform and analog-to-digital conversion, the modem circuitunit ex306 performs inverse spread spectrum processing of the data, andthe voice processing unit ex305 converts it into analog voice data, soas to output it via the voice output unit 208.

Furthermore, when transmitting e-mail in data communication mode, thetext data of the e-mail inputted by operating the operation keys ex204on the body unit is sent out to the main control unit ex311 via theoperation input control unit ex304. In the main control unit ex311,after the modem circuit unit ex306 performs spread spectrum processingof the text data and the send/receive circuit unit ex301 performsdigital-to-analog conversion and frequency transform for it, the data istransmitted to the base station ex110 via the antenna ex201.

When picture data is transmitted in data communication mode, the picturedata shot by the camera unit ex203 is supplied to the picture codingunit ex312 via the camera interface unit ex303. When it is nottransmitted, it is also possible to display the picture data shot by thecamera unit ex203 directly on the display unit 202 via the camerainterface unit ex303 and the LCD control unit ex302.

The picture coding unit ex312, which includes the moving picture codingapparatus as explained in the present invention, compresses and codesthe picture data supplied from the camera unit ex203 by the codingmethod used for the moving picture coding apparatus as shown in theabove embodiment so as to transform it into coded picture data, andsends it out to the multiplex/demultiplex unit ex308. At this time, themobile phone ex115 sends out the voices received by the voice input unitex205 during shooting by the camera unit ex203 to themultiplex/demultiplex unit ex308 as digital voice data via the voiceprocessing unit ex305.

The multiplex/demultiplex unit ex308 multiplexes the coded picture datasupplied from the picture coding unit ex312 and the voice data suppliedfrom the voice processing unit ex305 by a predetermined method, themodem circuit unit ex306 performs spread spectrum processing of themultiplexed data obtained as a result of the multiplexing, and thesend/receive circuit unit ex301 performs digital-to-analog conversionand frequency transform of the data for transmitting via the antennaex201.

As for receiving data of a moving picture file which is linked to a Webpage or the like in data communication mode, the modem circuit unitex306 performs inverse spread spectrum processing of the data receivedfrom the base station ex110 via the antenna ex201, and sends out themultiplexed data obtained as a result of the processing to themultiplex/demultiplex unit ex308.

In order to decode the multiplexed data received via the antenna ex201,the multiplex/demultiplex unit ex308 separates the multiplexed data intoa bit stream of picture data and a bit stream of voice data, andsupplies the coded picture data to the picture decoding unit ex309 andthe voice data to the voice processing unit ex305 respectively via thesynchronous bus ex313.

Next, the picture decoding unit ex309, which includes the moving picturedecoding apparatus as explained in the present invention, decodes thebit stream of picture data by the decoding method corresponding to thecoding method as shown in the above-mentioned embodiment to generatereproduced moving picture data, and supplies this data to the displayunit ex202 via the LCD control unit ex302, and thus moving picture dataincluded in a moving picture file linked to a Web page, for instance, isdisplayed. At the same time, the voice processing unit ex305 convertsthe voice data into analog voice data, and supplies this data to thevoice output unit ex208, and thus voice data included in a movingpicture file linked to a Web page, for instance, is reproduced.

The present invention is not limited to the above-mentioned system, andat least either the moving picture coding apparatus or the movingpicture decoding apparatus in the above-mentioned embodiment can beincorporated into a digital broadcasting system as shown in FIG. 21.Such ground-based or satellite digital broadcasting has been in the newslately. More specifically, a bit stream of video information istransmitted from a broadcast station ex409 to or communicated with abroadcast satellite ex410 via radio waves. Upon receipt of it, thebroadcast satellite ex410 transmits radio waves for broadcasting, ahome-use antenna ex406 with a satellite broadcast reception functionreceives the radio waves, and a television (receiver) ex401 or a set topbox (STB) ex407 decodes the bit stream for reproduction. The movingpicture decoding apparatus as shown in the above-mentioned embodimentcan be implemented in the reproduction device ex403 for reading off anddecoding the bit stream recorded on a storage medium ex402 that is arecording medium such as a CD and DVD. In this case, the reproducedvideo signals are displayed on a monitor ex404. It is also conceived toimplement the moving picture decoding apparatus in the set top box ex407connected to a cable ex405 for a cable television or the antenna ex406for satellite and/or ground-based broadcasting so as to reproduce themon a monitor ex408 of the television ex401. The moving picture decodingapparatus may be incorporated into the television, not in the set topbox. Or, a car ex412 having an antenna ex411 can receive signals fromthe satellite ex410 or the base station ex107 for reproducing movingpictures on a display device such as a car navigation system ex413.

Furthermore, the moving picture coding apparatus as shown in theabove-mentioned embodiment can code picture signals for recording on arecording medium. As a concrete example, there is a recorder ex420 suchas a DVD recorder for recording picture signals on a DVD disc ex421 anda disk recorder for recording them on a hard disk. They can be recordedon an SD card ex422. If the recorder ex420 includes the moving picturedecoding apparatus as shown in the above-mentioned embodiment, thepicture signals recorded on the DVD disc ex421 or the SD card ex422 canbe reproduced for display on the monitor ex408.

As the structure of the car navigation system ex413, the structurewithout the camera unit ex203, the camera interface unit ex303 and thepicture coding unit ex312, out of the units shown in FIG. 20, isconceivable. The same goes for the computer ex111, the television(receiver) ex401 and others.

In addition, three types of implementations can be conceived for aterminal such as the above-mentioned mobile phone ex114; asending/receiving terminal including both an encoder and a decoder, asending terminal including an encoder only, and a receiving terminalincluding a decoder only.

As described above, it is possible to use the moving picture codingmethod or the moving picture decoding method in the above-mentionedembodiments in any of the above-mentioned apparatus and system, andusing this method, the effects described in the above embodiments can beobtained.

Furthermore, the present invention is not limited to the aboveembodiments, but may be varied or modified in many ways without anydeparture from the scope of the present invention.

As described above, according to the moving picture coding method of thepresent invention, B-pictures can be coded using pictures which aretemporally close in display order as reference pictures. Accordingly,prediction efficiency for motion compensation is improved and thuscoding efficiency is improved.

In direct mode, by scaling a first motion vector of a second referencepicture, there is no need to transmit motion vector information and thusprediction efficiency can be improved.

Similarly, in direct mode, by scaling a first motion vectorsubstantially used for the direct mode coding of the second referencepicture, there is no need to transmit motion vector information, andprediction efficiency can be improved even if a co-located block in thesecond reference picture is coded in direct mode.

Also, in direct mode, by scaling a second motion vector which has beenused for coding a co-located block in a second reference picture, thereis no need to transmit motion vector information, and predictionefficiency can be improved even if the co-located block in the secondreference picture has only a second motion vector.

Furthermore, in direct mode, by setting forcedly a motion vector indirect mode to be “0”, when the direct mode is selected, there is noneed to transmit motion vector information nor to scale the motionvector, and thus processing volume can be reduced.

Also, in direct mode, by scaling a motion vector of a later P-picture,there is no need to store a motion vector of a second reference picturewhen the second reference picture is a B-picture. And, there is no needto transmit the motion vector information, and prediction efficiency canbe improved.

Furthermore, in direct mode, since a first motion vector is scaled if asecond reference picture has the first motion vector, and a secondmotion vector is scaled if the second reference picture does not havethe first motion vector but only the second motion vector, there is noneed to add motion vector information to a bit stream and predictionefficiency can be improved.

In addition, according to the moving picture decoding method of thepresent invention, a bit stream, which is generated as a result of interpicture bi-prediction coding using pictures which are located temporallyclose in display order as first and second reference pictures, can beproperly decoded.

INDUSTRIAL APPLICABILITY

As described above, the moving picture coding method and the movingpicture decoding method according to the present invention are useful asa method for coding picture data corresponding to pictures that form amoving picture to generate a bit stream, and a method for decoding thegenerated bit stream, using a mobile phone, a DVD apparatus and apersonal computer, for instance.

The invention claimed is:
 1. An integrated circuit comprising: an audioprocessing unit operable to process audio data; and a picture codingunit operable to code picture data, wherein said picture coding unitincludes: a coding unit operable to determine two motion vectors for acurrent block to be coded, based on one motion vector of a co-locatedblock which is a block included within a previously coded B-picture andco-located with the current block, and code the current block byperforming motion compensation on the current block in direct mode usingthe two motion vectors for the current block and two reference pictureswhich correspond to the two motion vectors for the current block,wherein said coding unit is operable to: when the co-located block hasbeen coded using two motion vectors and two reference pictures whichrespectively correspond to the two motion vectors of the co-locatedblock, specify, as a specified motion vector, one of the two motionvectors of the co-located block; generate the two motion vectors for thecurrent block that are used for performing direct mode motioncompensation on the current block, by scaling the specified motionvector by a ratio of a first difference and a second difference, thefirst difference being a difference between display order information ofa reference picture corresponding to the specified motion vector anddisplay order information of the previously coded B-picture includingthe co-located block, and the second difference being a differencebetween the display order information of the reference picturecorresponding to the specified motion vector and display orderinformation of the current picture including the current block; and codethe current block by performing motion compensation on the current blockin direct mode using the generated two motion vectors for the currentblock and two reference pictures, the two reference pictures being thereference picture corresponding to the specified motion vector and thepreviously coded B-picture including the co-located block, and the tworeference pictures respectively corresponding to the generated twomotion vectors, and wherein, when specifying the one of the two motionvectors of the co-located block as the specified motion vector, saidcoding unit is operable to specify only a forward motion vector as thespecified motion vector, from among the two motion vectors including theforward motion vector and a backward motion vector.
 2. A mobile terminalcomprising the integrated circuit according to claim
 1. 3. A codingapparatus comprising: an audio processing unit operable to process audiodata to be processed; a picture coding unit operable to code picturedata to be coded; and a power supply circuit configured to supply saidcoding apparatus with power, wherein said picture coding unit includes:a coding unit operable to determine two motion vectors for a currentblock to be coded, based on one motion vector of a co-located blockwhich is a block included within a previously coded B-picture andco-located with the current block, and code the current block byperforming motion compensation on the current block in direct mode usingthe two motion vectors for the current block and two reference pictureswhich correspond to the two motion vectors for the current block,wherein said coding unit is operable to: when the co-located block hasbeen coded using two motion vectors and two reference pictures whichrespectively correspond to the two motion vectors of the co-locatedblock, specify, as a specified motion vector, one of the two motionvectors of the co-located block; generate the two motion vectors for thecurrent block that are used for performing direct mode motioncompensation on the current block, by scaling the specified motionvector by a ratio of a first difference and a second difference, thefirst difference being a difference between display order information ofa reference picture corresponding to the specified motion vector anddisplay order information of the previously coded B-picture includingthe co-located block, and the second difference being a differencebetween the display order information of the reference picturecorresponding to the specified motion vector and display orderinformation of the current picture including the current block; and codethe current block by performing motion compensation on the current blockin direct mode using the generated two motion vectors for the currentblock and two reference pictures, the two reference pictures being thereference picture corresponding to the specified motion vector and thepreviously coded B-picture including the co-located block, and the tworeference pictures respectively corresponding to the generated twomotion vectors, and wherein, when specifying the one of the two motionvectors of the co-located block as the specified motion vector, saidcoding unit is operable to specify only a forward motion vector as thespecified motion vector, from among the two motion vectors including theforward motion vector and a backward motion vector.
 4. The codingapparatus according to claim 3, further comprising: a microphone thatinputs the audio data to be processed; and a camera that inputs thepicture data to be coded.